A study on computer vision for facial emotion recognition

• Published in: Scientific reports Vol. 13; no. 1; p. 8425
• Main Authors: Huang, Zi-Yu, Chiang, Chia-Chin, Chen, Jian-Hao, Chen, Yi-Chian, Chung, Hsin-Lung, Cai, Yu-Ping, Hsu, Hsiu-Chuan
• Format: Journal Article
• Language: English
• Published: London Nature Publishing Group UK 24.05.2023; Nature Publishing Group; Nature Portfolio
• Subjects: 639/766; 692/1537; 692/700; Accidental Injuries; Accuracy; Artificial Intelligence; Computer vision; Computers; Facial Recognition; Humanities and Social Sciences; Humans; multidisciplinary; Neural networks; Neural Networks, Computer; Pattern recognition; Science; Science (multidisciplinary)
• ISSN: 2045-2322; 2045-2322
• DOI: 10.1038/s41598-023-35446-4

Artificial intelligence has been successfully applied in various fields, one of which is computer vision. In this study, a deep neural network (DNN) was adopted for Facial emotion recognition (FER). One of the objectives in this study is to identify the critical facial features on which the DNN model focuses for FER. In particular, we utilized a convolutional neural network (CNN), the combination of squeeze-and-excitation network and the residual neural network, for the task of FER. We utilized AffectNet and the Real-World Affective Faces Database (RAF-DB) as the facial expression databases that provide learning samples for the CNN. The feature maps were extracted from the residual blocks for further analysis. Our analysis shows that the features around the nose and mouth are critical facial landmarks for the neural networks. Cross-database validations were conducted between the databases. The network model trained on AffectNet achieved 77.37% accuracy when validated on the RAF-DB, while the network model pretrained on AffectNet and then transfer learned on the RAF-DB results in validation accuracy of 83.37%. The outcomes of this study would improve the understanding of neural networks and assist with improving computer vision accuracy.