A Safety-Enhanced Reinforcement Learning-Based Decision-Making Method by the Dimensionality Reduction Monte Carlo Tree Search

• Published in: IEEE transactions on vehicular technology pp. 1 - 14
• Main Authors: Zhang, Lei, Cheng, Shuhui, Wang, Zhenpo, Liu, Jizheng, Wang, Mingqiang
• Format: Journal Article
• Language: English
• Published: IEEE 15.07.2024
• Subjects: Automated vehicles; Decision making; deep reinforcement learning; Hidden Markov models; partially observable Markov decision process; Planning; Safety; Switches; Trajectory; Turning; turning intention recognition
• DOI: 10.1109/TVT.2024.3424523
• ISSN: 0018-9545

Left-turning at unsignalized intersections poses significant challenges for automated vehicles. On this regard, Deep Reinforcement Learning (DRL) methods can achieve better traffic efficiency and success rate than rule-based methods, but they occasionally lead to collisions. This paper proposes a safety-enhanced method that integrates the DRL and the Dimensionality Reduction Monte Carlo Tree Search (DRMCTS) algorithm to achieve safety-enhanced trajectory planning at unsignalized intersections. First, DRMCTS is employed to address the partially observable Markov decision process problem. Through dimensionality reduction, it effectually enhances computational efficiency and problem-solving performance. Then a unified framework is introduced by simultaneously implementing DRL and the Gaussian Mixture Model Hidden Markov Model (GMM-HMM) in real-time. DRL determines actions in the current state while GMM-HMM identifies the turning intentions of surrounding vehicles (SVs). Under safe driving conditions, DRL makes decisions and outputs longitudinal acceleration with optimized ride comfort and traffic efficiency. When unsafe driving conditions are detected, DRMCTS would be activated to generate a collision-free trajectory to enhance the ego vehicle's driving safety. Through comprehensive simulations, the proposed scheme demonstrates superior traffic efficiency and reduced collision rates at unsignalized intersections with multiple SVs present.