Autonomous Tracking Using a Swarm of UAVs: A Constrained Multi-Agent Reinforcement Learning Approach

• Published in: IEEE Transactions on Vehicular Technology Vol. 69; no. 11; pp. 13702 - 13717
• Main Authors: Chen, Yu-Jia, Chang, Deng-Kai, Zhang, Cheng
• Format: Journal Article
• Language: English; Japanese
• Published: New York IEEE 01.11.2020; Institute of Electrical and Electronics Engineers (IEEE); The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; constrained Markov decision process; Feedback control; localization and tracking; Machine learning; Markov processes; Multi-agent reinforcement learning; Multiagent systems; Path planning; Radio frequency; Real-time systems; Robots; Sensors; Signal strength; Target tracking; Tracking systems; Unmanned aerial vehicles; unmanned aerial vehicles (UAVs)
• ISSN: 0018-9545; 1939-9359
• DOI: 10.1109/TVT.2020.3023733

In this paper, we aim to design an autonomous tracking system for a swarm of unmanned aerial vehicles (UAVs) to localize a radio frequency (RF) mobile target. In the system, UAVs equipped with omnidirectional received signal strength (RSS) sensors can cooperatively search the target with a specified tracking accuracy. To achieve fast localization and tracking in the highly dynamic channel environment (e.g., time-varying transmit power and intermittent signal), we formulate a flight decision problem as a constrained Markov decision process (CMDP) with the main objective of avoiding redundant UAV flight path. Then, we propose an enhanced multi-agent reinforcement learning to coordinate multiple UAVs performing real-time target tracking. The core of the proposed scheme is a feedback control system that takes into account the uncertainty of the channel estimate. We prove that the proposed algorithm can converge to the optimal decision. Our simulation results show that the proposed scheme outperforms standard Q-learning and multi-agent Q-learning algorithms in terms of searching time and successful localization probability.