Reinforcement learning based robot navigation using illegal actions for autonomous docking of surface vehicles in unknown environments

• Published in: Engineering applications of artificial intelligence Vol. 133; p. 108506
• Main Authors: Pereira, Maria Inês, Pinto, Andry Maykol
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.07.2024
• Subjects: Actor-critic; Autonomous docking; Autonomous surface vehicle; Reinforcement learning; Actor-critic; Autonomous docking; Autonomous surface vehicle; Reinforcement learning
• ISSN: 0952-1976; 1873-6769
• DOI: 10.1016/j.engappai.2024.108506

Autonomous Surface Vehicles (ASVs) are bound to play a fundamental role in the maintenance of offshore wind farms. Robust navigation for inspection vehicles should take into account the operation of docking within a harbouring structure, which is a critical and still unexplored maneuver. This work proposes an end-to-end docking approach for ASVs, based on Reinforcement Learning (RL), which teaches an agent to tackle collision-free navigation towards a target pose that allows the berthing of the vessel. The developed research presents a methodology that introduces the concept of illegal actions to facilitate the vessel’s exploration during the learning process. This method improves the adopted Actor-Critic (AC) framework by accelerating the agent’s optimization by approximately 38.02%. A set of comprehensive experiments demonstrate the accuracy and robustness of the presented method in scenarios with simulated environmental constraints (Beaufort Scale and Douglas Sea Scale), and a diversity of docking structures. Validation with two different real ASVs in both controlled and real environments demonstrates the ability of this method to enable safe docking maneuvers without prior knowledge of the scenario. •The novel concept of illegal actions to accelerate the optimization of an RL agent.•Situational awareness computed without prior knowledge of the scenario.•Autonomous docking of surface vehicles through an RL framework.•Extensive experiments in both simulated and real scenarios with two different vessels.