Multisource Transfer Learning for Cross-Subject EEG Emotion Recognition

• Published in: IEEE transactions on cybernetics Vol. 50; no. 7; pp. 3281 - 3293
• Main Authors: Li, Jinpeng, Qiu, Shuang, Shen, Yuan-Yuan, Liu, Cheng-Lin, He, Huiguang
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.07.2020; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Adult; Brain - physiology; Brain modeling; Brain-Computer Interfaces; Brain–computer interface; Calibration; Data models; Electroencephalography; Electroencephalography - classification; Electroencephalography - methods; Emotion recognition; Emotions; Emotions - classification; Humans; Learning; Machine Learning; Mapping; Pattern Recognition, Automated - methods; Temporal resolution; Training; Training data; transfer learning (TL); Young Adult
• ISSN: 2168-2267; 2168-2275; 2168-2275
• DOI: 10.1109/TCYB.2019.2904052

Electroencephalogram (EEG) has been widely used in emotion recognition due to its high temporal resolution and reliability. Since the individual differences of EEG are large, the emotion recognition models could not be shared across persons, and we need to collect new labeled data to train personal models for new users. In some applications, we hope to acquire models for new persons as fast as possible, and reduce the demand for the labeled data amount. To achieve this goal, we propose a multisource transfer learning method, where existing persons are sources, and the new person is the target. The target data are divided into calibration sessions for training and subsequent sessions for test. The first stage of the method is source selection aimed at locating appropriate sources. The second is style transfer mapping, which reduces the EEG differences between the target and each source. We use few labeled data in the calibration sessions to conduct source selection and style transfer. Finally, we integrate the source models to recognize emotions in the subsequent sessions. The experimental results show that the three-category classification accuracy on benchmark SEED improves by 12.72% comparing with the nontransfer method. Our method facilitates the fast deployment of emotion recognition models by reducing the reliance on the labeled data amount, which has practical significance especially in fast-deployment scenarios.