Deep reinforcement learning based control for Autonomous Vehicles in CARLA

• Published in: Multimedia tools and applications Vol. 81; no. 3; pp. 3553 - 3576
• Main Authors: Pérez-Gil, Óscar, Barea, Rafael, López-Guillén, Elena, Bergasa, Luis M., Gómez-Huélamo, Carlos, Gutiérrez, Rodrigo, Díaz-Díaz, Alejandro
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.01.2022; Springer Nature B.V
• Subjects: 1188: Artificial Intelligence for Physical Agents; Agents (artificial intelligence); Algorithms; Artificial intelligence; Autonomous vehicles; Computer Communication Networks; Computer Science; Data Structures and Information Theory; Deep learning; Machine learning; Multimedia Information Systems; Reagents; Route selection; Special Purpose and Application-Based Systems; Trajectories; DQN; DDPG; Deep Reinforcement Learning; Autonomous Vehicles; CARLA Simulator
• ISSN: 1380-7501; 1573-7721
• DOI: 10.1007/s11042-021-11437-3

Nowadays, Artificial Intelligence (AI) is growing by leaps and bounds in almost all fields of technology, and Autonomous Vehicles (AV) research is one more of them. This paper proposes the using of algorithms based on Deep Learning (DL) in the control layer of an autonomous vehicle. More specifically, Deep Reinforcement Learning (DRL) algorithms such as Deep Q-Network (DQN) and Deep Deterministic Policy Gradient (DDPG) are implemented in order to compare results between them. The aim of this work is to obtain a trained model, applying a DRL algorithm, able of sending control commands to the vehicle to navigate properly and efficiently following a determined route. In addition, for each of the algorithms, several agents are presented as a solution, so that each of these agents uses different data sources to achieve the vehicle control commands. For this purpose, an open-source simulator such as CARLA is used, providing to the system with the ability to perform a multitude of tests without any risk into an hyper-realistic urban simulation environment, something that is unthinkable in the real world. The results obtained show that both DQN and DDPG reach the goal, but DDPG obtains a better performance. DDPG perfoms trajectories very similar to classic controller as LQR. In both cases RMSE is lower than 0.1m following trajectories with a range 180-700m. To conclude, some conclusions and future works are commented.