SEEG Emotion Recognition Based on Transformer Network With Channel Selection and Explainability

• Published in: IEEE journal of biomedical and health informatics Vol. PP; pp. 1 - 11
• Main Authors: Yang, Zhuobin, Si, Xiaopeng, Jin, Weipeng, Huang, Dong, Zang, Yunliang, Yin, Shaoya, Ming, Dong
• Format: Journal Article
• Language: English
• Published: United States IEEE 03.07.2025
• Subjects: Accuracy; Brain; Brain modeling; Channel selection; Decoding; Deep learning; Electrodes; Electroencephalography; Emotion recognition; Epilepsy; Explainability; SEEG; Transformer; Transformers
• ISSN: 2168-2194; 2168-2208; 2168-2208
• DOI: 10.1109/JBHI.2025.3585528

Brain-computer interface (BCI) technology for emotion recognition holds significant potential for future applications in the treatment of refractory emotional disorders. Stereo-electroencephalography (SEEG), being less invasive, can precisely record neural activities originating from the cortex and the deep structures of the brain. Thus, it has broad application prospects in constructing emotion recognition BCI. In this study, SEEG data from nine subjects were collected to construct an emotion dataset, and a Spatial Transformer-based Hybrid Network (STHN) was proposed for SEEG emotion recognition. The triple-classification accuracy of STHN reached 83.56%, outperforming the baseline methods such as EEGNet, TSception, and the deep convolution neural network. Moreover, STHN can assign weights to each SEEG channel and select those channels that contribute more significantly to emotion recognition. It was found that when using the top 30% weighted SEEG channels as model inputs, the accuracy did not decrease significantly. Most of the channels with higher weights were located in brain regions strongly associated with emotions, such as the frontal lobe, the temporal lobe, and the hippocampus. This indicates that STHN is not merely a "black-box" model but possesses a degree of explainability. To the best of our knowledge, this is the first study to develop an SEEG emotion recognition algorithm, which is expected to play a crucial role in the monitoring and treatment of patients with refractory emotional disorders in the future.