Deductive Reinforcement Learning for Visual Autonomous Urban Driving Navigation

• Published in: IEEE transaction on neural networks and learning systems Vol. 32; no. 12; pp. 5379 - 5391
• Main Authors: Huang, Changxin, Zhang, Ronghui, Ouyang, Meizi, Wei, Pengxu, Lin, Junfan, Su, Jiang, Lin, Liang
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.12.2021; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Autonomous navigation; Autonomous urban driving; Autonomous vehicles; Coders; Computational modeling; Computer & video games; Decision making; Deduction; deductive reasoning; Deep learning; deep neural networks; Driving ability; Environment models; Navigation; Neural networks; Predictive models; Race cars; Reinforcement; Reinforcement learning; reinforcement learning (RL); Self-assessment; Task analysis; Trajectory; Vehicle safety; Visual discrimination learning
• ISSN: 2162-237X; 2162-2388; 2162-2388
• DOI: 10.1109/TNNLS.2021.3109284

Existing deep reinforcement learning (RL) are devoted to research applications on video games, e.g., The Open Racing Car Simulator (TORCS) and Atari games. However, it remains under-explored for vision-based autonomous urban driving navigation (VB-AUDN). VB-AUDN requires a sophisticated agent working safely in structured, changing, and unpredictable environments; otherwise, inappropriate operations may lead to irreversible or catastrophic damages. In this work, we propose a deductive RL (DeRL) to address this challenge. A deduction reasoner (DR) is introduced to endow the agent with ability to foresee the future and to promote policy learning. Specifically, DR first predicts future transitions through a parameterized environment model. Then, DR conducts self-assessment at the predicted trajectory to perceive the consequences of current policy resulting in a more reliable decision-making process. Additionally, a semantic encoder module (SEM) is designed to extract compact driving representation from the raw images, which is robust to the changes of the environment. Extensive experimental results demonstrate that DeRL outperforms the state-of-the-art model-free RL approaches on the public CAR Learning to Act (CARLA) benchmark and presents a superior performance on success rate and driving safety for goal-directed navigation.