EEG emotion recognition using fusion model of graph convolutional neural networks and LSTM

• Published in: Applied soft computing Vol. 100; p. 106954
• Main Authors: Yin, Yongqiang, Zheng, Xiangwei, Hu, Bin, Zhang, Yuang, Cui, Xinchun
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.03.2021
• Subjects: Differential entropy; EEG; Emotion recognition; Graph convolutional neural network; Long-short term memory neural network; Emotion recognition; Long-short term memory neural network; Graph convolutional neural network; EEG; Differential entropy
• ISSN: 1568-4946; 1872-9681
• DOI: 10.1016/j.asoc.2020.106954

In recent years, graph convolutional neural networks have become research focus and inspired new ideas for emotion recognition based on EEG. Deep learning has been widely used in emotion recognition, but it is still challenging to construct models and algorithms in practical applications. In this paper, we propose a novel emotion recognition method based on a novel deep learning model (ERDL). Firstly, EEG data is calibrated by 3s baseline data and divided into segments with 6s time window, and then differential entropy is extracted from each segment to construct feature cube. Secondly, the feature cube of each segment serves as input of the novel deep learning model which fuses graph convolutional neural network (GCNN) and long-short term memories neural networks (LSTM). In the fusion model, multiple GCNNs are applied to extract graph domain features while LSTM cells are used to memorize the change of the relationship between two channels within a specific time and extract temporal features, and Dense layer is used to attain the emotion classification results. At last, we conducted extensive experiments on DEAP dataset and experimental results demonstrate that the proposed method has better classification results than the state-of-the-art methods. We attained the average classification accuracy of 90.45% and 90.60% for valence and arousal in subject-dependent experiments while 84.81% and 85.27% in subject-independent experiments. •A fusion model of LSTM and GCNN for emotion classification is proposed.•Parallel GCNNs are constructed to extract graph domain features from each feature cube.•LSTM is utilized to memorize the relationship changes among EEG channels.