Safety Aware Autonomous Path Planning Using Model Predictive Reinforcement Learning for Inland Waterways

• Published in: IECON 2022 – 48th Annual Conference of the IEEE Industrial Electronics Society pp. 1 - 6
• Main Authors: Vanneste, Astrid, Vanneste, Simon, Vasseur, Olivier, Janssens, Robin, Billast, Mattias, Anwar, Ali, Mets, Kevin, De Schepper, Tom, Mercelis, Siegfried, Hellinckx, Peter
• Format: Conference Proceeding
• Language: English
• Published: IEEE 17.10.2022
• Subjects: autonomous path planning; autonomous shipping; collision avoidance; Navigation; Path planning; Planning; Predictive models; Reinforcement learning; Shape; Urban areas
• ISSN: 2577-1647
• DOI: 10.1109/IECON49645.2022.9968678

In recent years, interest in autonomous shipping in urban waterways has increased significantly due to the trend of keeping cars and trucks out of city centers. Classical approaches such as Frenet frame based planning and potential field navigation often require tuning of many configuration parameters and sometimes even require a different configuration depending on the situation. In this paper, we propose a novel path planning approach based on reinforcement learning called Model Predictive Reinforcement Learning (MPRL). MPRL calculates a series of waypoints for the vessel to follow. The environment is represented as an occupancy grid map, allowing us to deal with any shape of waterway and any number and shape of obstacles. We demonstrate our approach on two scenarios and compare the resulting path with path planning using a Frenet frame and path planning based on a proximal policy optimization (PPO) agent. Our results show that MPRL outperforms both baselines in both test scenarios. The PPO based approach was not able to reach the goal in either scenario while the Frenet frame approach failed in the scenario consisting of a corner with obstacles. MPRL was able to safely (collision free) navigate to the goal in both of the test scenarios.