GAPPO - A Graph Attention Reinforcement Learning based Robust Routing Algorithm

• Published in: 2023 IEEE 34th Annual International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC) pp. 1 - 7
• Main Authors: Li, Xinyuan, Xiao, Yang, Liu, Sixu, Lu, Xucong, Liu, Fang, Zhou, Wenli, Liu, Jun
• Format: Conference Proceeding
• Language: English
• Published: IEEE 05.09.2023
• Subjects: Attention; Deep Reinforcement Learning; Delays; Graph Neural Network; Heuristic algorithms; Measurement; Reinforcement learning; Robustness; Routing; Routing Algorithm; Topology
• ISSN: 2166-9589
• DOI: 10.1109/PIMRC56721.2023.10293938

Routing algorithms, which determine how to deliver traffic from the source to the destination, are essential for next-generation networks and the internet. To make optimal routing decisions in complex network environments, researchers have leveraged Deep Reinforcement Learning (DRL) to design next-generation routing mechanisms. However, most existing DRL-based routing algorithms are implemented using traditional Neural Networks (NN), which lack robustness against topology changes. In this paper, we propose a novel algorithm called GAPPO, which integrates Graph Attention Network (GAT) and Proximal Policy Optimization (PPO) to optimize routing policies with the objective of minimizing end-to-end (E2E) latency in networks affected by link failures. To evaluate the performance of the proposed algorithm, we conduct a series of experiments on dynamically changing topologies with link failures under different loads. Experimental results demonstrate that GAPPO outperforms benchmark algorithms in both simulated and real-world networks, confirming its powerful robustness against link failures.