SFT-SGAT: A semi-supervised fine-tuning self-supervised graph attention network for emotion recognition and consciousness detection

• Published in: Neural networks Vol. 180; p. 106643
• Main Authors: Qiu, Lina, Zhong, Liangquan, Li, Jianping, Feng, Weisen, Zhou, Chengju, Pan, Jiahui
• Format: Journal Article
• Language: English
• Published: United States Elsevier Ltd 01.12.2024
• Subjects: Adult; Attention - physiology; Brain-Computer Interfaces; Consciousness - physiology; Cross-subject; Electroencephalography - methods; Emotion recognition; Emotions - physiology; Female; Graph attention network; Humans; Male; Neural Networks, Computer; Self-supervised; Semi-supervised; Supervised Machine Learning; Emotion recognition; Self-supervised; Cross-subject; Graph attention network; Semi-supervised
• ISSN: 0893-6080; 1879-2782; 1879-2782
• DOI: 10.1016/j.neunet.2024.106643

Emotional recognition is highly important in the field of brain-computer interfaces (BCIs). However, due to the individual variability in electroencephalogram (EEG) signals and the challenges in obtaining accurate emotional labels, traditional methods have shown poor performance in cross-subject emotion recognition. In this study, we propose a cross-subject EEG emotion recognition method based on a semi-supervised fine-tuning self-supervised graph attention network (SFT-SGAT). First, we model multi-channel EEG signals by constructing a graph structure that dynamically captures the spatiotemporal topological features of EEG signals. Second, we employ a self-supervised graph attention neural network to facilitate model training, mitigating the impact of signal noise on the model. Finally, a semi-supervised approach is used to fine-tune the model, enhancing its generalization ability in cross-subject classification. By combining supervised and unsupervised learning techniques, the SFT-SGAT maximizes the utility of limited labeled data in EEG emotion recognition tasks, thereby enhancing the model’s performance. Experiments based on leave-one-subject-out cross-validation demonstrate that SFT-SGAT achieves state-of-the-art cross-subject emotion recognition performance on the SEED and SEED-IV datasets, with accuracies of 92.04% and 82.76%, respectively. Furthermore, experiments conducted on a self-collected dataset comprising ten healthy subjects and eight patients with disorders of consciousness (DOCs) revealed that the SFT-SGAT attains high classification performance in healthy subjects (maximum accuracy of 95.84%) and was successfully applied to DOC patients, with four patients achieving emotion recognition accuracies exceeding 60%. The experiments demonstrate the effectiveness of the proposed SFT-SGAT model in cross-subject EEG emotion recognition and its potential for assessing levels of consciousness in patients with DOC. •We propose a cross-subject emotion recognition method with state-of-the-art results.•We introduce a dynamic graph organization method to optimize graph structures.•We enhance emotion recognition by combining supervised and semi-supervised training.