Multi-source joint domain adaptation for cross-subject and cross-session emotion recognition from electroencephalography

• Published in: Frontiers in human neuroscience Vol. 16; p. 921346
• Main Authors: Liang, Shengjin, Su, Lei, Fu, Yunfa, Wu, Liping
• Format: Journal Article
• Language: English
• Published: Lausanne Frontiers Research Foundation 15.09.2022; Frontiers Media S.A
• Subjects: affective brain–computer interface; affective computing; Brain research; Classification; domain adaptation; EEG; Electroencephalography; emotion recognition; Emotions; Experiments; House brands; Human-computer interaction; Interfaces; Neural networks; Neuroscience; User experience
• ISSN: 1662-5161; 1662-5161
• DOI: 10.3389/fnhum.2022.921346

As an important component to promote the development of affective brain–computer interfaces, the study of emotion recognition based on electroencephalography (EEG) has encountered a difficult challenge; the distribution of EEG data changes among different subjects and at different time periods. Domain adaptation methods can effectively alleviate the generalization problem of EEG emotion recognition models. However, most of them treat multiple source domains, with significantly different distributions, as one single source domain, and only adapt the cross-domain marginal distribution while ignoring the joint distribution difference between the domains. To gain the advantages of multiple source distributions, and better match the distributions of the source and target domains, this paper proposes a novel multi-source joint domain adaptation (MSJDA) network. We first map all domains to a shared feature space and then align the joint distributions of the further extracted private representations and the corresponding classification predictions for each pair of source and target domains. Extensive cross-subject and cross-session experiments on the benchmark dataset, SEED, demonstrate the effectiveness of the proposed model, where more significant classification results are obtained on the more difficult cross-subject emotion recognition task.