FlexDATE: Flexible and Disturbance-Aware Traffic Engineering With Reinforcement Learning in Software-Defined Networks

• Published in: IEEE/ACM transactions on networking Vol. 31; no. 4; pp. 1 - 16
• Main Authors: Ye, Minghao, Zhang, Junjie, Guo, Zehua, Chao, H. Jonathan
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.08.2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Critical flow; Degradation; Graph neural networks; Linear programming; Machine learning; Measurement; network disturbance; Network topologies; Network topology; Optimization; Quality of service; Quality of service architectures; Reinforcement learning; Rerouteing; Routing; Software engineering; Software-defined networking; Traffic control; Traffic engineering
• ISSN: 1063-6692; 1558-2566
• DOI: 10.1109/TNET.2022.3217083

Traffic Engineering (TE) is an important network operation that routes/reroutes flows based on network topology and traffic demands to optimize network performance. Recently, new emerging applications pose challenges to TE with dynamic network conditions, where frequent routing updates are required to maintain good network performance with Software-Defined Networking (SDN). However, flow rerouting operations could lead to considerable Quality of Service (QoS) degradation and service disruption, which is often neglected by existing TE solutions. In this paper, we apply a new QoS metric named network disturbance to measure the negative impact of flow rerouting operations performed by TE. To achieve near-optimal load balancing performance and mitigate network disturbance together in dynamic network scenarios, we propose a flexible and disturbance-aware TE solution called that combines Reinforcement Learning (RL) and Linear Programming (LP). Specifically, leverages RL to intelligently identify flexible numbers of critical flows for each traffic matrix and reroutes these critical flows based on LP optimization to improve network performance with low disturbance. Empowered by a customized actor-critic architecture coupled with Graph Neural Networks (GNNs), can generalize well to unseen traffic scenarios and remain resilient to single link failures. Extensive simulations are conducted on five real-world network topologies to evaluate with real and synthetic traffic traces. The results show that can achieve the performance target (i.e., 90% of optimal performance) in 99% of network scenarios and effectively mitigate the average and maximum network disturbance by up to 9.1% and 38.6%, respectively, compared to state-of-the-art TE solutions.