Predictive trajectory planning for autonomous vehicles at intersections using reinforcement learning

• Published in: Transportation research. Part C, Emerging technologies Vol. 149; p. 104063
• Main Authors: Zhang, Ethan, Zhang, Ruixuan, Masoud, Neda
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.04.2023
• Subjects: Autonomous vehicles; Reinforcement learning; Trajectory planning; Trajectory prediction; Trajectory prediction; Trajectory planning; Autonomous vehicles; Reinforcement learning
• ISSN: 0968-090X; 1879-2359
• DOI: 10.1016/j.trc.2023.104063

In this work we put forward a predictive trajectory planning framework to help autonomous vehicles plan future trajectories. We develop a partially observable Markov decision process (POMDP) to model this sequential decision making problem, and a deep reinforcement learning solution methodology to learn high-quality policies. The POMDP model utilizes driving scenarios, condensed into graphs, as inputs. More specifically, an input graph contains information on the history trajectory of the subject vehicle, predicted trajectories of other agents in the scene (e.g., other vehicles, pedestrians, and cyclists), as well as predicted risk levels posed by surrounding vehicles to devise safe, comfortable, and energy-efficient trajectories for the subject vehicle to follow. In order to obtain sufficient driving scenarios to use as training data, we propose a simulation framework to generate socially acceptable driving scenarios using a real world autonomous vehicle dataset. The simulation framework utilizes Bayesian Gaussian mixture models to learn trajectory patterns of different agent types, and Gibbs sampling to ensure that the distribution of simulated scenarios matches that of the real-world dataset collected by an autonomous fleet. We evaluate the proposed work in two complex urban driving environments: a non-signalized T-junction and a non-signalized lane merge intersection. Both environments provide vastly more complex driving scenarios compared to a highway driving environment, which has been mostly the focus of previous studies. The framework demonstrates promising performance for planning horizons as long as five seconds. We compare safety, comfort, and energy efficiency of the planned trajectories against human-driven trajectories in both experimental driving environments, and demonstrate that it outperforms human-driven trajectories in a statistically significant fashion in all aspects. •A predictive–prescriptive trajectory planning framework.•Planned trajectories account for safety, comfort, and fuel efficiency.•Graph-based reinforcement learning is used to obtain a policy.•Planned trajectories outperform human-driven trajectories.