EmT: A Novel Transformer for Generalized Cross-Subject EEG Emotion Recognition

• Published in: IEEE transaction on neural networks and learning systems Vol. 36; no. 6; pp. 10381 - 10393
• Main Authors: Ding, Yi, Tong, Chengxuan, Zhang, Shuailei, Jiang, Muyun, Li, Yong, Lim, Kevin JunLiang, Guan, Cuntai
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.06.2025
• Subjects: Algorithms; Brain modeling; Cognitive processes; Data mining; Databases, Factual; Deep learning; Electroencephalography; electroencephalography (EEG); Electroencephalography - methods; Emotion recognition; Emotions - classification; Emotions - physiology; Feature extraction; Graph neural networks; Humans; Learning systems; Mixers; Neural Networks, Computer; transformer; Transformers
• ISSN: 2162-237X; 2162-2388; 2162-2388
• DOI: 10.1109/TNNLS.2025.3552603

Integrating prior knowledge of neurophysiology into neural network architecture enhances the performance of emotion decoding. While numerous techniques emphasize learning spatial and short-term temporal patterns, there has been a limited emphasis on capturing the vital long-term contextual information associated with emotional cognitive processes. In order to address this discrepancy, we introduce a novel transformer model called emotion transformer (EmT). EmT is designed to excel in both generalized cross-subject electroencephalography (EEG) emotion classification and regression tasks. In EmT, EEG signals are transformed into a temporal graph format, creating a sequence of EEG feature graphs using a temporal graph construction (TGC) module. A novel residual multiview pyramid graph convolutional neural network (RMPG) module is then proposed to learn dynamic graph representations for each EEG feature graph within the series, and the learned representations of each graph are fused into one token. Furthermore, we design a temporal contextual transformer (TCT) module with two types of token mixers to learn the temporal contextual information. Finally, the task-specific output (TSO) module generates the desired outputs. Experiments on four publicly available datasets show that EmT achieves higher results than the baseline methods for both EEG emotion classification and regression tasks. The code is available at https://github.com/yi-ding-cs/EmT .