Optimal robust formation control for heterogeneous multi‐agent systems based on reinforcement learning

• Published in: International journal of robust and nonlinear control Vol. 32; no. 5; pp. 2683 - 2704
• Main Authors: Yan, Bing, Shi, Peng, Lim, Cheng‐Chew, Shi, Zhiyuan
• Format: Journal Article
• Language: English
• Published: Bognor Regis Wiley Subscription Services, Inc 25.03.2022
• Subjects: adaptive observer; Algorithms; heterogeneous multi‐agent systems; Internal model principle; Learning; Optimization; reinforcement learning; Robust control; robust formation control; Unmanned aerial vehicles; Unmanned ground vehicles; Wildfires
• ISSN: 1049-8923; 1099-1239
• DOI: 10.1002/rnc.5828

In this article, a reinforcement learning (RL)‐based robust control strategy is proposed for uncertain heterogeneous multi‐agent systems to achieve optimal collision‐free time‐varying formations. Without using any global information, a fully distributed adaptive observer is developed to estimate both dynamics and states of the reference and disturbance systems. The observer parameters are found by an observed model‐based or a model‐free off‐policy RL algorithm. Using the internal model principle, a novel optimal robust formation control strategy is developed based on another proposed off‐policy RL algorithm. The algorithm addresses the nonquadratic optimization problem when the system model is completely unknown. Taking the bushfire edge tracking and patrolling task for an unmanned aerial vehicle‐unmanned ground vehicle heterogeneous system as an example, the effectiveness and robustness of the developed control strategy are verified by simulations.