EEG-Based Cross-Subject Driver Drowsiness Recognition With an Interpretable Convolutional Neural Network

• Published in: IEEE transaction on neural networks and learning systems Vol. 34; no. 10; pp. 7921 - 7933
• Main Authors: Cui, Jian, Lan, Zirui, Sourina, Olga, Muller-Wittig, Wolfgang
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.10.2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Accuracy; Artificial neural networks; Automobiles; Brain modeling; Convolutional neural network (CNN); Convolutional neural networks; Deep learning; depthwise separable convolution; driver drowsiness recognition; Driver fatigue; Drowsiness; EEG; Electroencephalography; electroencephalography (EEG); interpretable deep learning; interpretation techniques; Machine learning; Monitoring; Neural networks; Recognition; Sleepiness; Teaching methods; Vehicles
• DOI: 10.1109/TNNLS.2022.3147208
• ISSN: 2162-237X; 2162-2388

In the context of electroencephalogram (EEG)-based driver drowsiness recognition, it is still challenging to design a calibration-free system, since EEG signals vary significantly among different subjects and recording sessions. Many efforts have been made to use deep learning methods for mental state recognition from EEG signals. However, existing work mostly treats deep learning models as black-box classifiers, while what have been learned by the models and to which extent they are affected by the noise in EEG data are still underexplored. In this article, we develop a novel convolutional neural network combined with an interpretation technique that allows sample-wise analysis of important features for classification. The network has a compact structure and takes advantage of separable convolutions to process the EEG signals in a spatial-temporal sequence. Results show that the model achieves an average accuracy of 78.35% on 11 subjects for leave-one-out cross-subject drowsiness recognition, which is higher than the conventional baseline methods of 53.40%-72.68% and state-of-the-art deep learning methods of 71.75%-75.19%. Interpretation results indicate the model has learned to recognize biologically meaningful features from EEG signals, e.g., alpha spindles, as strong indicators of drowsiness across different subjects. In addition, we also explore reasons behind some wrongly classified samples with the interpretation technique and discuss potential ways to improve the recognition accuracy. Our work illustrates a promising direction on using interpretable deep learning models to discover meaningful patterns related to different mental states from complex EEG signals.