Deep Reinforcement Learning for UAV Trajectory Design Considering Mobile Ground Users

• Published in: Sensors (Basel, Switzerland) Vol. 21; no. 24; p. 8239
• Main Authors: Lee, Wonseok, Jeon, Young, Kim, Taejoon, Kim, Young-Il
• Format: Journal Article
• Language: English
• Published: Switzerland MDPI AG 09.12.2021; MDPI
• Subjects: Algorithms; Altitude; Bandwidths; Communication; Communications systems; Deep learning; Disaster relief; Energy consumption; Line of sight; Optimization; Power gain; Radio equipment; reinforcement learning; Trajectory optimization; Unmanned aerial vehicles; unmanned aerial vehicles; trajectory optimization; reinforcement learning
• ISSN: 1424-8220; 1424-8220
• DOI: 10.3390/s21248239

A network composed of unmanned aerial vehicles (UAVs), serving as base stations (UAV-BS network), is emerging as a promising component in next-generation communication systems. In the UAV-BS network, the optimal positioning of a UAV-BS is an essential requirement to establish line-of-sight (LoS) links for ground users. A novel deep Q-network (DQN)-based learning model enabling the optimal deployment of a UAV-BS is proposed. Moreover, without re-learning of the model and the acquisition of the path information of ground users, the proposed model presents the optimal UAV-BS trajectory while ground users move. Specifically, the proposed model optimizes the trajectory of a UAV-BS by maximizing the mean opinion score (MOS) for ground users who move to various paths. Furthermore, the proposed model is highly practical because, instead of the locations of individual mobile users, an average channel power gain is used as an input parameter. The accuracy of the proposed model is validated by comparing the results of the model with those of a mathematical optimization solver.