Classification of Driver Cognitive Load in Conditionally Automated Driving: Utilizing Electrocardiogram-Based Spectrogram with Lightweight Neural Network

• Published in: Transportation research record Vol. 2678; no. 12; pp. 1560 - 1573
• Main Authors: Shi, Wenxin, Wang, Zuyuan, Wang, Ange, He, Dengbo
• Format: Journal Article
• Language: English
• Published: Los Angeles, CA SAGE Publications 01.12.2024
• Subjects: Driver State Estimation ECG signal; Automated Driving; Cognitive Load; Spectrogram
• ISSN: 0361-1981; 2169-4052
• DOI: 10.1177/03611981241252797

With the development of conditionally automated driving, drivers will be allowed to perform non-driving-related tasks. Under such circumstances, continuous monitoring of driver cognitive load will play an increasingly important role in ensuring that drivers have sufficient mental resources to take over control of the vehicle should the driving automation fail. However, estimation of cognitive load is challenging because of the difficulties in identifying high-level feature representation and accounting for interindividual differences. Physiological measures are believed to be promising candidates for cognitive load estimation in partially automated vehicles. However, current estimation methods are mainly based on the manual feature extraction of time- or frequency-domain indicators from physiological signals, which may not adapt to dynamic driving conditions. With the development of deep learning, the neural network has shown good performance in automatically capturing high-level features from input data. Inspired by this, we adopted a novel approach to classify driver cognitive load based on electrocardiogram (ECG) spectrograms, in which the driver’s ECG signal was collected and transformed into a 2D spectrogram by a short-time Fourier transform. A squeeze-and-excitation network-based deep-learning framework that can capture high-level features and pays more attention to the cognition-related features of the spectrogram was proposed for classification. Experiments on a publicly available dataset demonstrated that our model achieved an accuracy of 96.76% in differentiating two levels of cognitive load for a within-subject evaluation and 71.50% accuracy with an across-subjects evaluation. The results demonstrated the feasibility of detecting drivers’ cognitive load through deep learning using ECG spectrogram alone.