A Fast Approach to Satellite Range Rescheduling Using Deep Reinforcement Learning

• Published in: IEEE transactions on aerospace and electronic systems Vol. 59; no. 6; pp. 9390 - 9403
• Main Authors: Liang, Jun, Liu, Jian-Ping, Sun, Qing, Zhu, Yue-He, Zhang, Yi-Chuan, Song, Jian-Guo, He, Bo-Yong
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.12.2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Aerodynamics; Deep learning; Dynamic programming; Dynamic scheduling; fast solvers; Indexes; intelligent systems; Low earth orbit satellites; Mathematical models; Missions; Optimization; Rescheduling; Run time (computers); Satellite constellations; satellite tracking; Satellites; Scheduling
• ISSN: 0018-9251; 1557-9603
• DOI: 10.1109/TAES.2023.3317356

Real-time emergency event processing for mega-constellation management requires an approach that can quickly obtain a satisfactory satellite range rescheduling scheme. In this article, a deep reinforcement learning (DRL) approach is proposed for rapid rescheduling on the basis of established satellite range scheduling schemes. Three rescheduling principles with different impacts on established scheduling schemes are first analyzed. A mathematical model for both maximizing the completion profits of dynamic missions and minimizing their impacts on scheduled missions is formulated. An efficient DRL-based approach to satellite range rescheduling is then developed. A mask processing mechanism is integrated into the reinforcement learning framework to effectively eliminate the conflicts accurately expressed by the conflict space, and a modified pointer network (Ptr-Net) based on the mask processing mechanism is employed to approximate the action-value function for optimal decision-making. Experimental results show that the proposed approach significantly outperforms the rule-based heuristic and another DRL-based method in terms of computational efficiency and optimization performance, and the total time spent during the training and testing phases of the proposed approach is approximately 13% of the total running time required by the search-based methods.