FLDNet: Frame-Level Distilling Neural Network for EEG Emotion Recognition

• Published in: IEEE journal of biomedical and health informatics Vol. 25; no. 7; pp. 2533 - 2544
• Main Authors: Wang, Zhe, Gu, Tianhao, Zhu, Yiwen, Li, Dongdong, Yang, Hai, Du, Wenli
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.07.2021; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Benchmarks; Brain modeling; Correlation; Deep learning; Distillation; EEG; EEG emotion recognition; Electroencephalography; Emotion recognition; Emotions; Feature extraction; Feature recognition; Frames; Information processing; Knowledge distillation; Logic gates; Machine learning; neural network; Neural networks; self-attention; teacher-student structure; Training
• ISSN: 2168-2194; 2168-2208; 2168-2208
• DOI: 10.1109/JBHI.2021.3049119

Based on the current research on EEG emotion recognition, there are some limitations, such as hand-engineered features, redundant and meaningless signal frames and the loss of frame-to-frame correlation. In this paper, a novel deep learning framework is proposed, named the frame-level distilling neural network (FLDNet), for learning distilled features from the correlations of different frames. A layer named the frame gate is designed to integrate weighted semantic information on multiple frames to remove redundant and meaningless signal frames. A triple-net structure is introduced to distill the learned features net by net to replace the hand-engineered features with professional knowledge. Specifically, one neural network is normally trained for several epochs. Then, a second network of the same structure will be initialized again to learn the extracted features from the frame gate of the first neural network based on the output of the first net. Similarly, the third net improves the features based on the frame gate of the second network. To utilize the representation ability of the triple neural network, an ensemble layer is conducted to integrate the discriminative ability of the proposed framework for final decisions. Consequently, the proposed FLDNet provides an effective method for capturing the correlation between different frames and automatically learn distilled high-level features for emotion recognition. The experiments are carried out in a subject-independent emotion recognition task on public emotion datasets of DEAP and DREAMER benchmarks, which have demonstrated the effectiveness and robustness of the proposed FLDNet.