Strategies for Scaleable Communication and Coordination in Multi-Agent (UAV) Systems

• Published in: Aerospace Vol. 9; no. 9; p. 488
• Main Authors: Ponniah, Jonathan, Dantsker, Or D.
• Format: Journal Article
• Language: English
• Published: Basel MDPI AG 01.09.2022
• Subjects: Autonomous underwater vehicles; Clustering; Communication; Communications systems; Computer architecture; distributed systems; Flying-machines; Learning; mobile ad-hoc networks; multi-agent systems; Multiagent systems; reinforcement learning; Robotics; Robots; Surveillance; tracking; Unmanned aerial vehicles; Wireless networks; New York
• ISSN: 2226-4310; 2226-4310
• DOI: 10.3390/aerospace9090488

A system is considered in which agents (UAVs) must cooperatively discover interest-points (i.e., burning trees, geographical features) evolving over a grid. The objective is to locate as many interest-points as possible in the shortest possible time frame. There are two main problems: a control problem, where agents must collectively determine the optimal action, and a communication problem, where agents must share their local states and infer a common global state. Both problems become intractable when the number of agents is large. This survey/concept paper curates a broad selection of work in the literature pointing to a possible solution; a unified control/communication architecture within the framework of reinforcement learning. Two components of this architecture are locally interactive structure in the state-space, and hierarchical multi-level clustering for system-wide communication. The former mitigates the complexity of the control problem and the latter adapts to fundamental throughput constraints in wireless networks. The challenges of applying reinforcement learning to multi-agent systems are discussed. The role of clustering is explored in multi-agent communication. Research directions are suggested to unify these components.