From Regional to Global Brain: A Novel Hierarchical Spatial-Temporal Neural Network Model for EEG Emotion Recognition

• Published in: IEEE transactions on affective computing Vol. 13; no. 2; pp. 568 - 578
• Main Authors: Li, Yang, Zheng, Wenming, Wang, Lei, Zong, Yuan, Cui, Zhen
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.04.2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Biological neural networks; Brain; Brain modeling; Classifiers; Computational modeling; Domains; EEG emotion recognition; Electrodes; Electroencephalography; Emotion recognition; Emotions; Feature extraction; Machine learning; Neural networks; regional to global; spatial-temporal network
• DOI: 10.1109/TAFFC.2019.2922912
• ISSN: 1949-3045

In this paper, we propose a novel Electroencephalograph (EEG) emotion recognition method inspired by neuroscience with respect to the brain response to different emotions. The proposed method, denoted by R2G-STNN, consists of spatial and temporal neural network models with regional to global hierarchical feature learning process to learn discriminative spatial-temporal EEG features. To learn the spatial features, a bidirectional long short term memory (BiLSTM) network is adopted to capture the intrinsic spatial relationships of EEG electrodes within brain region and between brain regions, respectively. Considering that different brain regions play different roles in the EEG emotion recognition, a region-attention layer into the R2G-STNN model is also introduced to learn a set of weights to strengthen or weaken the contributions of brain regions. Based on the spatial feature sequences, BiLSTM is adopted to learn both regional and global spatial-temporal features and the features are fitted into a classifier layer for learning emotion-discriminative features, in which a domain discriminator working corporately with the classifier is used to decrease the domain shift between training and testing data. Finally, to evaluate the proposed method, we conduct both subject-dependent and subject-independent EEG emotion recognition experiments on SEED database, and the experimental results show that the proposed method achieves state-of-the-art performance.