Driver Mental Fatigue Detection Based on Head Posture Using New Modified reLU-BiLSTM Deep Neural Network

• Published in: IEEE transactions on intelligent transportation systems Vol. 23; no. 8; pp. 10957 - 10969
• Main Authors: Ansari, Shahzeb, Naghdy, Fazel, Du, Haiping, Pahnwar, Yasmeen Naz
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.08.2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Angular acceleration; Artificial neural networks; Biomedical monitoring; Computer vision; Crashes; deep learning model; Driver behavior; Driver behaviour; Fatigue; Head movement; head posture; Intelligent vehicles; Machine learning; Magnetic heads; mental fatigue detection; Monitoring; Motion capture; Neural networks; reLU-BiLSTM; Series (mathematics); Vehicle dynamics; Vehicles
• DOI: 10.1109/TITS.2021.3098309
• ISSN: 1524-9050

Early detection of driver mental fatigue is one of the active areas of research in smart and intelligent vehicles. There are various methods, based on measuring the physiological characteristics of the driver utilising sensors and computer vision, proposed in the literature. In general, driver behaviour is unpredictable that can suddenly change the nature of driving and dynamics under mental fatigue. This results in sudden variations in driver body posture and head movement, with consequent inattentive behaviour that can end in fatal accidents and crashes. In the process of delineating the different driving patterns of driver states while active or influenced by mental fatigue, this paper contributes to advancing direct measurement approaches. In the novel approach proposed in this paper, driver mental fatigue and drowsiness are measured by monitoring driver's head posture motions using XSENS motion capture system. The experiments were conducted on 15 healthy subjects on a MATHWORKS driver-in-loop (DIL) simulator, interfaced with Unreal Engine 4 studio. A new modified bidirectional long short-term memory deep neural network, based on a rectified linear unit layer, was designed, trained and tested on 3D time-series head angular acceleration data for sequence-to-sequence classification. The results showed that the proposed classifier outperformed state-of-art approaches and conventional machine learning tools, and successfully recognised driver's active, fatigue and transition states, with overall training accuracy of 99.2%, sensitivity of 97.54%, precision and F1 scores of 97.38% and 97.46%, respectively. The limitations of the current work and directions for future work are also explored.