Emotion Recognition Using Continuous Wavelet Transform and Ensemble of Convolutional Neural Networks through Transfer Learning from Electroencephalogram Signal

• Published in: Frontiers in biomedical technologies Vol. 10; no. 1
• Main Authors: Bagherzadeh, Sara, Maghooli, Keivan, Shalbaf, Ahmad, Maghsoudi, Arash
• Format: Journal Article
• Language: English
• Published: Tehran University of Medical Sciences 01.12.2023
• Subjects: Continuous Wavelet Transform; Deep Learning; Electroencephalogram; Emotion Recognition; Ensemble Approach; Transfer Learning
• ISSN: 2345-5837; 2345-5837
• DOI: 10.18502/fbt.v10i1.11512

Purpose: Emotions are integral brain states that can influence our behavior, decision-making, and functions. Electroencephalogram (EEG) is an appropriate modality for emotion recognition since it has high temporal resolution and is a non-invasive and cheap technique. Materials and Methods: A novel approach based on Ensemble pre-trained Convolutional Neural Networks (ECNNs) is proposed to recognize four emotional classes from EEG channels of individuals watching music video clips. First, scalograms are built from one-dimensional EEG signals by applying the Continuous Wavelet Transform (CWT) method. Then, these images are used to re-train five CNNs: AlexNet, VGG-19, Inception-v1, ResNet-18, and Inception-v3. Then, the majority voting method is applied to make the final decision about emotional classes. The 10-fold cross-validation method is used to evaluate the performance of the proposed method on EEG signals of 32 subjects from the DEAP database. Results:.The experiments showed that applying the proposed ensemble approach in combinations of scalograms of frontal and parietal regions improved results. The best accuracy, sensitivity, precision, and F-score to recognize four emotional states achieved 96.90% ± 0.52, 97.30 ± 0.55, 96.97 ± 0.62, and 96.74 ± 0.56, respectively. Conclusion: So, the newly proposed model from EEG signals improves recognition of the four emotional states in the DEAP database.