The Identification of Non-Driving Activities with Associated Implication on the Take-Over Process

• Published in: Sensors (Basel, Switzerland) Vol. 22; no. 1; p. 42
• Main Authors: Yang, Lichao, Babayi Semiromi, Mahdi, Xing, Yang, Lv, Chen, Brighton, James, Zhao, Yifan
• Format: Journal Article
• Language: English
• Published: Switzerland MDPI AG 22.12.2021; MDPI
• Subjects: 3D CNN; Accidents, Traffic; Automation; Automobile Driving; Awareness; Driver behavior; Fatalities; Humans; level 3 automation; non-driving related activity (NDRA) classification; Questionnaires; Reading; Recognition, Psychology; situation awareness; Situational awareness; take-over transition; Workloads; United States > US; 3D CNN; take-over transition; non-driving related activity (NDRA) classification; level 3 automation; situation awareness
• ISSN: 1424-8220; 1424-8220
• DOI: 10.3390/s22010042

In conditionally automated driving, the engagement of non-driving activities (NDAs) can be regarded as the main factor that affects the driver's take-over performance, the investigation of which is of great importance to the design of an intelligent human-machine interface for a safe and smooth control transition. This paper introduces a 3D convolutional neural network-based system to recognize six types of driver behaviour (four types of NDAs and two types of driving activities) through two video feeds based on head and hand movement. Based on the interaction of driver and object, the selected NDAs are divided into active mode and passive mode. The proposed recognition system achieves 85.87% accuracy for the classification of six activities. The impact of NDAs on the perspective of the driver's situation awareness and take-over quality in terms of both activity type and interaction mode is further investigated. The results show that at a similar level of achieved maximum lateral error, the engagement of NDAs demands more time for drivers to accomplish the control transition, especially for the active mode NDAs engagement, which is more mentally demanding and reduces drivers' sensitiveness to the driving situation change. Moreover, the haptic feedback torque from the steering wheel could help to reduce the time of the transition process, which can be regarded as a productive assistance system for the take-over process.