Core-Brain-Network-Based Multilayer Convolutional Neural Network for Emotion Recognition

• Published in: IEEE transactions on instrumentation and measurement Vol. 70; pp. 1 - 9
• Main Authors: Gao, Zhongke, Li, Rumei, Ma, Chao, Rui, Linge, Sun, Xinlin
• Format: Journal Article
• Language: English
• Published: New York IEEE 2021; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Artificial neural networks; Brain; Brain network; Continuous wavelet transform; Continuous wavelet transforms; convolutional neural network (CNN); Correlation; Deep learning; differential entropy (DE); electroencephalogram (EEG) signals; Electroencephalography; Emotion recognition; Emotional factors; Emotions; Entropy; Feature extraction; Frequencies; Multilayers; Neural networks; Spearman correlation coefficient; Wavelet transforms
• ISSN: 0018-9456; 1557-9662
• DOI: 10.1109/TIM.2021.3090164

In this article, we propose a method for emotion classification based on multilayer convolutional neural network (MCNN) and combining differential entropy (DE) and brain network. First, we use continuous wavelet transform (CWT) to get the time-frequency representation (TFR) of electroencephalogram (EEG) signals on each channel and extract rich information from different frequency bands for subsequent analysis. Brain networks are then constructed in multiple bands to characterize the spatial connections hidden in the multichannel EEG signals. Based on brain networks, we further develop core brain networks through a set of key nodes determined by DE. These core brain networks are associated with brain activities and differ markedly between different emotional states. The final designed MCNN model takes DE features and core brain networks as inputs for emotion recognition. We evaluate our method on the SJTU emotion EEG dataset, and the average accuracy of 15 subjects achieves 91.45%. Utilizing the complementary features of DE and brain network, the proposed method provides an efficient framework for accurate emotion recognition from EEG signals.