PR-PL: A Novel Prototypical Representation Based Pairwise Learning Framework for Emotion Recognition Using EEG Signals

• Published in: IEEE transactions on affective computing Vol. 15; no. 2; pp. 657 - 670
• Main Authors: Zhou, Rushuang, Zhang, Zhiguo, Fu, Hong, Zhang, Li, Li, Linling, Huang, Gan, Li, Fali, Yang, Xin, Dong, Yining, Zhang, Yuan-Ting, Liang, Zhen
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.04.2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Adaptation models; affective brain-computer interface; Affective computing; Brain modeling; Computational modeling; EEG; Electroencephalography; Emotion recognition; Emotions; Human-computer interface; Labeling; Labels; Learning; Representations; Transfer learning
• ISSN: 1949-3045; 1949-3045
• DOI: 10.1109/TAFFC.2023.3288118

Affective brain-computer interface based on electroencephalography (EEG) is an important branch in the field of affective computing. However, the individual differences in EEG emotional data and the noisy labeling problem in the subjective feedback seriously limit the effectiveness and generalizability of existing models. To tackle these two critical issues, we propose a novel transfer learning framework with Prototypical Representation based Pairwise Learning ( PR-PL ). The discriminative and generalized EEG features are learned for emotion revealing across individuals and the emotion recognition task is formulated as pairwise learning for improving the model tolerance to the noisy labels. More specifically, a prototypical learning is developed to encode the inherent emotion-related semantic structure of EEG data and align the individuals' EEG features to a shared common feature space under consideration of the feature separability of both source and target domains. Based on the aligned feature representations, pairwise learning with an adaptive pseudo labeling method is introduced to encode the proximity relationships among samples and alleviate the label noises effect on modeling. Extensive results on two benchmark databases (SEED and SEED-IV) under four different cross-validation evaluation protocols validate the model reliability and stability across subjects and sessions. Compared to the literature, the average enhancement of emotion recognition across four different evaluation protocols is 2.04% (SEED) and 2.58% (SEED-IV).