Adaptive multi-branch CNN of integrating manual features and functional network for driver fatigue detection

• Published in: Biomedical signal processing and control Vol. 102; p. 107262
• Main Authors: Yang, Kun, Zhang, Keze, Hu, Yubin, Xu, Jing, Yang, Bing, Kong, Wanzeng, Zhang, Jianhai
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.04.2025
• Subjects: Cross-subject; Driver fatigue; EEG; Functional network; Graph convolutional network; Driver fatigue; Graph convolutional network; Cross-subject; EEG; Functional network
• ISSN: 1746-8094
• DOI: 10.1016/j.bspc.2024.107262

The use of deep learning techniques to detect driver fatigue from EEG data has received much attention. However, most existing studies based on deep learning rely on a single type of information. The effect of integrating different types of information (or features) based on deep learning techniques is unknown. This paper proposed an adaptive multi-branch CNN model (i.e. adMBCNN) to innovatively integrate the functional network and various handcrafted features for driver fatigue detection. The adMBCNN model contains four branches, where one CNN branch adequately and effectively exploits the functional connectivity between electrodes and the other three CNN branches learn deep information from three kinds of handcrafted features. The adMBCNN model can automatically adapt to the data characteristics to extract discriminative information from the functional network and multiple types of handcrafted features, thus overcoming the weakness of CNNs in constructing and learning channel relationships, but also avoiding the shortcomings of typical GCN-based methods in extracting valid information from the functional network. The experimental results on two publicly available datasets showed that the proposed adMBCNN model outperforms the existing complex methods and achieved achieves the best performance. The idea and its implementation in this paper provide a novel and successful perspective for the application of deep learning techniques to other EEG-based tasks. •This paper proposed an adaptive multi-branch CNN model (i.e. adMBCNN) to innovatively integrate the manual functional network and various handcrafted features for driver fatigue detection.•The paper shows that the typical GCN-based method is difficult to extract valid information from the functional network. But, the proposed adMBCNN model can extract effective discriminative information from the functional network.•This paper skilfully introduces the brain functional network and various handcrafted features to overcome the CNN disadvantage in constructing and exploiting the channel relationship.