A Deep Reinforcement Learning-Based Dynamic Traffic Offloading in Space-Air-Ground Integrated Networks (SAGIN)

• Published in: IEEE journal on selected areas in communications Vol. 40; no. 1; pp. 276 - 289
• Main Authors: Tang, Fengxiao, Hofner, Hans, Kato, Nei, Kaneko, Kazuma, Yamashita, Yasutaka, Hangai, Masatake
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.01.2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Bandwidth; Channel allocation; Communications traffic; Deep learning; double Q-learning; Heuristic algorithms; Machine learning; Nodes; Proposals; Reinforcement learning; reinforcement learning (RL); satellite communication; Satellites; Space-air-ground integrated networks (SAGIN); Topology; traffic offloading; UAV
• ISSN: 0733-8716; 1558-0008
• DOI: 10.1109/JSAC.2021.3126073

Space-Air-Ground Integrated Networks (SAGIN) is considered as the key structure of the next generation network. The space satellites and air nodes are the potential candidates to assist and offload the terrain transmissions. However, due to the high mobility of space and air nodes as well as the high dynamic of network traffic, the conventional traffic offloading strategy is not applicable for the high dynamic SAGIN. In this paper, we propose a reinforcement learning based traffic offloading for SAGIN by considering the high mobility of nodes as well as frequent changing network traffic and link state. In the proposal, a double Q-learning algorithm with improved delay-sensitive replay memory algorithm (DSRPM) is proposed to train the node to decide offloading strategy based on the local and neighboring historical information. Furthermore, a joint information collection with hello package and offline training mechanism is proposed to assist the proposed offloading algorithm. The simulation shows that the proposal outperforms conventional offloading algorithms in terms of signaling overhead, dynamic adaptivity, packet drop rate and transmission delay.