EEG-based emotion recognition using 4D convolutional recurrent neural network

• Published in: Cognitive neurodynamics Vol. 14; no. 6; pp. 815 - 828
• Main Authors: Shen, Fangyao, Dai, Guojun, Lin, Guang, Zhang, Jianhai, Kong, Wanzeng, Zeng, Hong
• Format: Journal Article
• Language: English
• Published: Dordrecht Springer Netherlands 01.12.2020; Springer Nature B.V
• Subjects: Artificial Intelligence; Artificial neural networks; Biochemistry; Biomedical and Life Sciences; Biomedicine; Cognitive Psychology; Computer Science; Deep learning; EEG; Electrodes; Electroencephalography; Emotion recognition; Emotions; Frequency dependence; Long short-term memory; Methods; Neural networks; Neurosciences; Normal distribution; Physiology; Random variables; Recurrent neural networks; Research Article; Spatial data; Convolutional recurrent neural network; Emotion recognition; 4D data; EEG
• ISSN: 1871-4080; 1871-4099
• DOI: 10.1007/s11571-020-09634-1

In this paper, we present a novel method, called four-dimensional convolutional recurrent neural network, which integrating frequency, spatial and temporal information of multichannel EEG signals explicitly to improve EEG-based emotion recognition accuracy. First, to maintain these three kinds of information of EEG, we transform the differential entropy features from different channels into 4D structures to train the deep model. Then, we introduce CRNN model, which is combined by convolutional neural network (CNN) and recurrent neural network with long short term memory (LSTM) cell. CNN is used to learn frequency and spatial information from each temporal slice of 4D inputs, and LSTM is used to extract temporal dependence from CNN outputs. The output of the last node of LSTM performs classification. Our model achieves state-of-the-art performance both on SEED and DEAP datasets under intra-subject splitting. The experimental results demonstrate the effectiveness of integrating frequency, spatial and temporal information of EEG for emotion recognition.