Understanding common human driving semantics for autonomous vehicles

• Published in: Patterns (New York, N.Y.) Vol. 4; no. 7; p. 100730
• Main Authors: Xia, Yingji, Geng, Maosi, Chen, Yong, Sun, Sudan, Liao, Chenlei, Zhu, Zheng, Li, Zhihui, Ochieng, Washington Yotto, Angeloudis, Panagiotis, Elhajj, Mireille, Zhang, Lei, Zeng, Zhenyu, Zhang, Bing, Gao, Ziyou, Chen, Xiqun (Michael)
• Format: Journal Article
• Language: English
• Published: United States Elsevier Inc 14.07.2023; Elsevier
• Subjects: autonomous vehicle; driving behavior perception; driving semantics; electroencephalography; hierarchical understanding abstraction; human-machine interaction; neural-informed model; neuroscience; driving semantics; electroencephalography; human-machine interaction; hierarchical understanding abstraction; neuroscience; driving behavior perception; autonomous vehicle; neural-informed model; DSML 2: Proof-of-concept: Data science output has been formulated, implemented, and tested for one domain/problem
• ISSN: 2666-3899; 2666-3899
• DOI: 10.1016/j.patter.2023.100730

Autonomous vehicles will share roads with human-driven vehicles until the transition to fully autonomous transport systems is complete. The critical challenge of improving mutual understanding between both vehicle types cannot be addressed only by feeding extensive driving data into data-driven models but by enabling autonomous vehicles to understand and apply common driving behaviors analogous to human drivers. Therefore, we designed and conducted two electroencephalography experiments for comparing the cerebral activities of human linguistics and driving understanding. The results showed that driving activates hierarchical neural functions in the auditory cortex, which is analogous to abstraction in linguistic understanding. Subsequently, we proposed a neural-informed, semantics-driven framework to understand common human driving behavior in a brain-inspired manner. This study highlights the pathway of fusing neuroscience into complex human behavior understanding tasks and provides a computational neural model to understand human driving behaviors, which will enable autonomous vehicles to perceive and think like human drivers. •Reveal human auditory cortex activation during driving•Discover the hierarchical structure of human driving understanding•Propose a neural-informed semantics-driven driving understanding model•Address long-term contextual dependency of driving behaviors “Driving like humans” is the ultimate goal of autonomous driving. Hence, human-like driving understanding ability is required for autonomous vehicles to better understand the driving behaviors of surrounding human-driven vehicles. In this study, we investigated human driving neural response and subsequently built a biologically plausible model to interpret driving behaviors like humans. This study pioneers the design of bio-inspired, human-like autonomous vehicles and can ultimately benefit future research of human-machine interactions. Autonomous vehicles will share roads with human-driven vehicles and bring with them problems regarding bidirectional understanding of driving behavior. Based on cerebral neurological findings from the human process for understanding driving, a novel neural-inspired semantics-driven driving understanding model is proposed for autonomous vehicles. The model imitates the way humans understand driving and can interpret long-term driving behavior evolutions like human drivers.