An Efficient LSTM Network for Emotion Recognition From Multichannel EEG Signals

• Published in: IEEE transactions on affective computing Vol. 13; no. 3; pp. 1528 - 1540
• Main Authors: Du, Xiaobing, Ma, Cuixia, Zhang, Guanhua, Li, Jinyao, Lai, Yu-Kun, Zhao, Guozhen, Deng, Xiaoming, Liu, Yong-Jin, Wang, Hongan
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.07.2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: attention mechanism; Brain modeling; Channels; Data models; domain adaptation; Domains; Electrodes; Electroencephalography; Emotion recognition; Emotions; Feature extraction; Frequency-domain analysis; LSTM; Machine learning; multichannel EEG
• ISSN: 1949-3045; 1949-3045
• DOI: 10.1109/TAFFC.2020.3013711

Most previous EEG-based emotion recognition methods studied hand-crafted EEG features extracted from different electrodes. In this article, we study the relation among different EEG electrodes and propose a deep learning method to automatically extract the spatial features that characterize the functional relation between EEG signals at different electrodes. Our proposed deep model is called AT tention-based LSTM with D omain D iscriminator (ATDD-LSTM), a model based on Long Short-Term Memory (LSTM) for emotion recognition that can characterize nonlinear relations among EEG signals of different electrodes. To achieve state-of-the-art emotion recognition performance, the architecture of ATDD-LSTM has two distinguishing characteristics: (1) By applying the attention mechanism to the feature vectors produced by LSTM, ATDD-LSTM automatically selects suitable EEG channels for emotion recognition, which makes the learned model concentrate on the emotion related channels in response to a given emotion; (2) To minimize the significant feature distribution shift between different sessions and/or subjects, ATDD-LSTM uses a domain discriminator to modify the data representation space and generate domain-invariant features. We evaluate the proposed ATDD-LSTM model on three public EEG emotional databases (DEAP, SEED and CMEED) for emotion recognition. The experimental results demonstrate that our ATDD-LSTM model achieves superior performance on subject-dependent (for the same subject), subject-independent (for different subjects) and cross-session (for the same subject) evaluation.