Facial expression recognition based on Electroencephalogram and facial landmark localization

• Published in: Technology and health care Vol. 27; no. 4; pp. 373 - 387
• Main Authors: Li, Dahua, Wang, Zhe, Gao, Qiang, Song, Yu, Yu, Xiao, Wang, Chuhan
• Format: Journal Article
• Language: English
• Published: London, England SAGE Publications 01.01.2019; Sage Publications Ltd
• Subjects: Accuracy; Affective computing; Algorithms; Discrete Wavelet Transform; EEG; Electroencephalography; Electroencephalography - methods; Energy; Face recognition; Facial Expression; Facial Recognition; Humans; Image reconstruction; Localization; Mental disorders; Mental Disorders - diagnosis; Pattern recognition; Rehabilitation; Sensitivity and Specificity; Signal processing; Signal Processing, Computer-Assisted; Support Vector Machine; Support vector machines; Wavelet Analysis; Wavelet transforms; discrete wavelet transform (DWT); facial landmark localization; Electroencephalogram (EEG); fusion feature vector; support vector machine (SVM)
• ISSN: 0928-7329; 1878-7401; 1878-7401
• DOI: 10.3233/THC-181538

BACKGROUND: Facial expression recognition plays an essential role in affective computing, mental illness diagnosis and rehabilitation. Therefore, facial expression recognition has attracted more and more attention over the years. OBJECTIVE: The goal of this paper was to improve the accuracy of the Electroencephalogram (EEG)-based facial expression recognition. METHODS: In this paper, we proposed a fusion facial expression recognition method based on EEG and facial landmark localization. The EEG signal processing and facial landmark localization are the two key parts. The raw EEG signals is preprocessed by discrete wavelet transform (DWT). The energy feature vector is composed of energy features of the reconstructed signal. For facial landmark localization, images of the subjects’ facial expression are processed by facial landmark localization, and the facial features are calculated by landmarks of essence. In this research, we fused the energy feature vector and facial feature vector, and classified the fusion feature vector with the support vector machine (SVM). RESULTS: From the experiments, we found that the accuracy of facial expression recognition was increased 4.16% by fusion method (86.94 ± 4.35%) than EEG-based facial expression recognition (82.78 ± 5.78%). CONCLUSION: The proposed method obtain a higher accuracy and a stronger generalization capability.