TSception: Capturing Temporal Dynamics and Spatial Asymmetry From EEG for Emotion Recognition

• Published in: IEEE transactions on affective computing Vol. 14; no. 3; pp. 2238 - 2250
• Main Authors: Ding, Yi, Robinson, Neethu, Zhang, Su, Zeng, Qiuhao, Guan, Cuntai
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.07.2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Artificial neural networks; Asymmetry; Brain; Convolution; Convolutional neural networks; Deep learning; Electroencephalography; Emotion recognition; Emotions; Feature extraction; Kernel; Machine learning; Representations; Temporal resolution; Unsupervised learning
• ISSN: 1949-3045; 1949-3045
• DOI: 10.1109/TAFFC.2022.3169001

The high temporal resolution and the asymmetric spatial activations are essential attributes of electroencephalogram (EEG) underlying emotional processes in the brain. To learn the temporal dynamics and spatial asymmetry of EEG towards accurate and generalized emotion recognition, we propose TSception, a multi-scale convolutional neural network that can classify emotions from EEG. TSception consists of dynamic temporal, asymmetric spatial, and high-level fusion layers, which learn discriminative representations in the time and channel dimensions simultaneously. The dynamic temporal layer consists of multi-scale 1D convolutional kernels whose lengths are related to the sampling rate of EEG, which learns the dynamic temporal and frequency representations of EEG. The asymmetric spatial layer takes advantage of the asymmetric EEG patterns for emotion, learning the discriminative global and hemisphere representations. The learned spatial representations will be fused by a high-level fusion layer. Using more generalized cross-validation settings, the proposed method is evaluated on two publicly available datasets DEAP and MAHNOB-HCI. The performance of the proposed network is compared with prior reported methods such as SVM, KNN, FBFgMDM, FBTSC, Unsupervised learning, DeepConvNet, ShallowConvNet, and EEGNet. TSception achieves higher classification accuracies and F1 scores than other methods in most of the experiments. The codes are available at: https://github.com/yi-ding-cs/TSception