Sparse Bandit Learning Based Location Management for Space-Ground Integrated Networks

• Published in: IEEE transactions on vehicular technology Vol. 72; no. 8; pp. 10314 - 10329
• Main Authors: He, Huasen, Qin, Changkun, Chen, Shuangwu, Jiang, Xiaofeng, Yang, Jian, Hanzo, Lajos
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.08.2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Bandits theory; Caching; Delays; Handover; Heuristic algorithms; IP networks; location management; Low earth orbits; Machine learning; Mapping; mapping allocation; Multi-armed bandit problems; Optimization; Protocols; reinforcement learning; Resource management; Satellites; Storage capacity; Upper bounds
• ISSN: 0018-9545; 1939-9359
• DOI: 10.1109/TVT.2023.3258140

The Space-Ground Integrated Network (SGIN) concept constitutes a promising solution for providing seamless global coverage. However, the mobility of satellites and wireless terminals imposes unprecedented challenges on the location management in SGIN. We tackle this challenge by conceiving a split identifier (ID) and Network Address (NA) based design for providing natural mobility support, and characterize the ID-NA mapping allocation problem by exploiting the storage capacity of both Geostationary Earth Orbit Satellites (GEOSs) and Low Earth Orbiting Satellites (LEOSs) to form a spatially distributed binding resolution system and optimize the caching reward in each LEOS. By considering the large quantity of ID-NA mapping and the sparsity of popular mapping having positive mean caching rewards, we formulate the mapping allocation problem as a sparse Multi-Armed Bandit (MAB) learning procedure, where the mappings are treated as the arms and the LEOSs act as the players. A distributed learning algorithm, namely the Sparse Upper confidence bound based Learning aided Caching algorithm (SULC), is proposed for estimating the mean caching rewards of mappings and selecting the optimal mappings for caching. Moreover, we derive a sub-linear upper bound of the cumulative learning regret to prove the learning efficiency of the proposed SULC. Extensive simulations have been conducted to show that the proposed SULC can quickly identify the popular mappings and provide near-optimal content hit rates. In contrast with the existing solutions, SULC has higher caching rewards and can significantly reduce the cumulative regret after a short period of learning.