Semi-Supervised Dual-Stream Self-Attentive Adversarial Graph Contrastive Learning for Cross-Subject EEG-Based Emotion Recognition

• Published in: IEEE transactions on affective computing Vol. 16; no. 1; pp. 290 - 305
• Main Authors: Ye, Weishan, Zhang, Zhiguo, Teng, Fei, Zhang, Min, Wang, Jianhong, Ni, Dong, Li, Fali, Xu, Peng, Liang, Zhen
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.01.2025; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Brain modeling; Data models; domain adaption; EEG; Effectiveness; Electroencephalography; Emotion recognition; Emotions; Feature extraction; graph contrastive learning; Graphical representations; Labels; Learning; semi-supervised learning; Streams; Transfer learning
• ISSN: 1949-3045; 1949-3045
• DOI: 10.1109/TAFFC.2024.3433470

Electroencephalography (EEG) is an objective tool for emotion recognition with promising applications. However, the scarcity of labeled data remains a major challenge in this field, limiting the widespread use of EEG-based emotion recognition. In this paper, a semi-supervised D ual-stream S elf-attentive A dversarial G raph C ontrastive learning framework (termed as DS-AGC ) is proposed to tackle the challenge of limited labeled data in cross-subject EEG-based emotion recognition. The DS-AGC framework includes two parallel streams for extracting non-structural and structural EEG features. The non-structural stream incorporates a semi-supervised multi-domain adaptation method to alleviate distribution discrepancy among labeled source domain, unlabeled source domain, and unknown target domain. The structural stream develops a graph contrastive learning method to extract effective graph-based feature representation from multiple EEG channels in a semi-supervised manner. Further, a self-attentive fusion module is developed for feature fusion, sample selection, and emotion recognition, which highlights EEG features more relevant to emotions and data samples in the labeled source domain that are closer to the target domain. Extensive experiments are conducted on four benchmark databases (SEED, SEED-IV, SEED-V, and FACED) using a semi-supervised cross-subject leave-one-subject-out cross-validation evaluation protocol. The results show that the proposed model outperforms existing methods under different incomplete label conditions with an average improvement of 2.17%, which demonstrates its effectiveness in addressing the label scarcity problem in cross-subject EEG-based emotion recognition.