Self-supervised EEG emotion recognition models based on CNN

• Published in: IEEE transactions on neural systems and rehabilitation engineering Vol. 31; p. 1
• Main Authors: Wang, Xingyi, Ma, Yuliang, Cammon, Jared, Fang, Feng, Gao, Yunyuan, Zhang, Yingchun
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.01.2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Artificial neural networks; Brain; Brain modeling; Classification; Computer applications; Convolution; Convolutional neural networks; EEG; Electroencephalography; Electroencephalography - methods; emotion classification; Emotion recognition; Emotions; Feature extraction; Human-computer interface; Humans; Implants; Learning algorithms; Machine Learning; Multi-Task Learning; Neural networks; Neural Networks, Computer; Resource utilization; Self-supervised; Self-supervised learning; Signal classification; Task analysis; Transformations
• ISSN: 1534-4320; 1558-0210
• DOI: 10.1109/TNSRE.2023.3263570

Emotion plays crucial roles in human life. Recently, emotion classification from electroencephalogram (EEG) signal has attracted attention by researchers due to the rapid development of brain computer interface (BCI) techniques and machine learning algorithms. However, recent studies on emotion classification show resource utilization because they use the fully-supervised learning methods. Therefore, in this study, we applied the self-supervised learning methods to improve the efficiency of resources usage. We employed a self-supervised approach to train deep multi-task convolutional neural network (CNN) for EEG-based emotion classification. First, six signal transformations were performed on unlabeled EEG data to construct the pretext task. Second, a multi-task CNN was used to perform signal transformation recognition on the transformed signals together with the original signals. After the signal transformation recognition network was trained, the convolutional layer network was frozen and the fully connected layer was reconstructed as emotion recognition network. Finally, the EEG data with affective labels were used to train the emotion recognition network to clarify the emotion. In this paper, we conduct extensive experiments from the data scaling perspective using the SEED, DEAP affective dataset. Results showed that the self-supervised learning methods can learn the internal representation of data and save computation time compared to the fully-supervised learning methods. In conclusion, our study suggests that the self-supervised machine learning model can improve the performance of emotion classification compared to the conventional fully supervised model.