Network Selection Based on Evolutionary Game and Deep Reinforcement Learning in Space-Air-Ground Integrated Network

• Published in: IEEE transactions on network science and engineering Vol. 9; no. 3; pp. 1802 - 1812
• Main Authors: Fan, Kexin, Feng, Bowen, Zhang, Xilin, Zhang, Qinyu
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.05.2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Communications systems; Decision making; Deep reinforcement learning; Evolutionary algorithms; evolutionary game; Game theory; Games; Heuristic algorithms; Machine learning; network selection; Optimization; Resource utilization; Satellites; Sociology; space-air-ground integrated network; Statistics; Strategy; Throughput
• ISSN: 2327-4697; 2334-329X
• DOI: 10.1109/TNSE.2022.3153480

In next generation communication system, space-air-ground integrated network (SAGIN) would be utilized to provide ubiquitous and unlimited wireless connectivity with large coverage, high throughput, and strong resilience. In this integrated network, there are multiple heterogeneous network options to satisfy service requirements, where an efficient network selection strategy is required to improve resource utilization and achieve load balance. In this paper, we propose a system model of network selection in SAGIN and formulate a corresponding evolutionary game. A network selection algorithm based on evolutionary game is proposed to study the autonomous decision-making process of network selection as a supplement. We also propose a deep deterministic policy gradient (DDPG)-based network selection algorithm to handle continuous and high-dimensional action spaces. A particular case is studied for further simulation and analysis. The evolutionary game obtains the selection strategy with the highest payoff at the evolutionary equilibrium point, and the stability of evolutionary equilibrium is proved by varying relative factors. The DDPG-based network selection algorithm obtains the same strategy with the highest reward at convergence at a slower speed. In comprehensive comparison, our proposed algorithms perform better than the repeated stochastic game approach and proximal policy optimization (PPO) algorithm.