Reinforcement learning-based close formation control for underactuated surface vehicle with prescribed performance and time-varying state constraints

• Published in: Ocean engineering Vol. 256; p. 111361
• Main Authors: Chen, Huizi, Yan, Huaicheng, Wang, Yueying, Xie, Shaorong, Zhang, Dan
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 15.07.2022
• Subjects: Actor-critic neural network; Close formation control; Finite-time sliding mode control; Prescribed formation performance; Reinforcement learning; Time-varying state constraints; Finite-time sliding mode control; Time-varying state constraints; Prescribed formation performance; Close formation control; Actor-critic neural network; Reinforcement learning
• ISSN: 0029-8018; 1873-5258
• DOI: 10.1016/j.oceaneng.2022.111361

This paper studies close formation control problem with prescribed performance and time-varying state constraints for a group of 4-degrees-of-freedom (DOF) underactuated surface vehicles (USVs) subject to actuator faults, input saturation and input delay. A finite-time sliding mode control (SMC) scheme based on reinforcement learning (RL) algorithm is introduced to guarantee prescribed formation performance without violating velocity error constraints. By using actor-critic neural network (NN)-based RL algorithm, the actuator faults and system uncertainties are accurately estimated. Afterwards, an exponential decreasing boundary function is developed to suppress overshoot more reasonably, and a novel mechanism of switching gain is given to alleviate chattering inherent in SMC while the RL-based compensation term is constructed to handle the formation accuracy problem caused by the reduced switching gain. Besides, auxiliary nonlinear continuous function and Pade approximation have been successfully applied to process actuator saturation and input delay, respectively. Numerical simulations and experimental results are exhibited to verify the effectiveness and superior formation performance of the proposed control method. •The sway-yaw error subsystem stabilization and roll reduction in close formation can be achieved simultaneously through the same control input (e.g. a single rudder).•A novel prescribed boundary function is developed, which helps USVs formation avoid overlarge initial error by determining the maximum convergence time.•Compared with the published SMC strategies, the RL-based SMC control strategy is utilized to achieve chattering suppression while maintaining a high-precision formation performance.