Multi-UAV Dynamic Wireless Networking With Deep Reinforcement Learning

• Published in: IEEE communications letters Vol. 23; no. 12; pp. 2243 - 2246
• Main Authors: Wang, Qiang, Zhang, Wenqi, Liu, Yuanwei, Liu, Ying
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.12.2019; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Capacity; Computer simulation; Constraints; deep reinforcement learning; Downlink; Drones; Machine learning; Markov analysis; Markov processes; movement; Neural networks; Optimization; Real time; Real-time systems; Reinforcement learning; Unmanned aerial vehicles; Wireless communication systems; Wireless communications; Wireless networks
• ISSN: 1089-7798; 1558-2558
• DOI: 10.1109/LCOMM.2019.2940191

This letter investigates a novel unmanned aerial vehicle (UAV)-enabled wireless communication system, where multiple UAVs transmit information to multiple ground terminals (GTs). We study how the UAVs can optimally employ their mobility to maximize the real-time downlink capacity while covering all GTs. The system capacity is characterized, by optimizing the UAV locations subject to the coverage constraint. We formula the UAV movement problem as a Constrained Markov Decision Process (CMDP) problem and employ Q-learning to solve the UAV movement problem. Since the state of the UAV movement problem has large dimensions, we propose Dueling Deep Q-network (DDQN) algorithm which introduces neural networks and dueling structure into Q-learning. Simulation results demonstrate the proposed movement algorithm is able to track the movement of GTs and obtains real-time optimal capacity, subject to coverage constraint.