Intelligent collision avoidance algorithms for USVs via deep reinforcement learning under COLREGs

• Published in: Ocean engineering Vol. 217; p. 107704
• Main Authors: Xu, Xinli, Lu, Yu, Liu, Xiaocheng, Zhang, Weidong
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.12.2020
• Subjects: COLREGs; Deep learning; Dynamic collision avoidance; Reinforcement learning; Unmanned surface vehicles; Deep learning; COLREGs; Unmanned surface vehicles; Reinforcement learning; Dynamic collision avoidance
• ISSN: 0029-8018; 1873-5258
• DOI: 10.1016/j.oceaneng.2020.107704

In the field of unmanned surface vehicles, intelligent collision avoidance technology is essential to ensure the safety of navigating. In this paper, the problem of avoiding moving boats for USVs under the constraints of COLREGs is studied. A COLREGs intelligent collision avoidance (CICA) algorithm based on deep reinforcement learning is proposed, which can automatically extract state features by using powerful deep neural networks. The reward function is designed, which ensures that the USV navigates to the target while obeying COLREGs to avoid dynamic obstacles. A method is proposed to track the current network weight to update the target network weight, which improves the stability of the algorithm in learning the optimal strategy. It is shown that the CICA algorithm converges with fewer training times through ε-greedy with both decaying ε and reward threshold than other three strategies. By comparing the CICA algorithm with the artificial potential field method and the velocity obstacle method, it is concluded that the CICA algorithm is superior to the other two algorithms. •A intelligent collision avoidance algorithm for underactuated USVs is proposed based on deep reinforcement learning.•The USV has the ability to avoid dynamic obstacles under the control of this algorithm, and obeys COLREGs.•The algorithm can deal with complex scenarios without defining a complete collision avoidance control law.