RL-Routing: An SDN Routing Algorithm Based on Deep Reinforcement Learning

• Published in: IEEE transactions on network science and engineering Vol. 7; no. 4; pp. 3185 - 3199
• Main Authors: Chen, Yi-Ren, Rezapour, Amir, Tzeng, Wen-Guey, Tsai, Shi-Chun
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.10.2020; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Cognitive sdn; Communication networks; Control systems; deep reinforcement learning; Delays; Heuristic algorithms; Machine learning; Mathematical models; Network topologies; Network topology; Reinforcement learning; Routing; routing algorithm; software defined networks; Switches; Throughput; Traffic control; Traffic engineering
• ISSN: 2327-4697; 2334-329X
• DOI: 10.1109/TNSE.2020.3017751

Communication networks are difficult to model and predict because they have become very sophisticated and dynamic. We develop a reinforcement learning routing algorithm (RL-Routing) to solve a traffic engineering (TE) problem of SDN in terms of throughput and delay. RL-Routing solves the TE problem via experience, instead of building an accurate mathematical model. We consider comprehensive network information for state representation and use one-to-many network configuration for routing choices. Our reward function, which uses network throughput and delay, is adjustable for optimizing either upward or downward network throughput. After appropriate training, the agent learns a policy that predicts future behavior of the underlying network and suggests better routing paths between switches. The simulation results show that RL-Routing obtains higher rewards and enables a host to transfer a large file faster than Open Shortest Path First (OSPF) and Least Loaded (LL) routing algorithms on various network topologies. For example, on the NSFNet topology, the sum of rewards obtained by RL-Routing is 119.30, whereas those of OSPF and LL are 106.59 and 74.76, respectively. The average transmission time for a 40GB file using RL-Routing is <inline-formula><tex-math notation="LaTeX">\text{25.2}~s</tex-math></inline-formula>. Those of OSPF and LL are <inline-formula><tex-math notation="LaTeX">\text{63}~s</tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX">\text{53.4}~s</tex-math></inline-formula>, respectively.