Deep learning-based facial emotion recognition for human–computer interaction applications

• Published in: Neural computing & applications Vol. 35; no. 32; pp. 23311 - 23328
• Main Authors: Chowdary, M. Kalpana, Nguyen, Tu N., Hemanth, D. Jude
• Format: Journal Article
• Language: English
• Published: London Springer London 01.11.2023; Springer Nature B.V
• Subjects: Artificial Intelligence; Classification; Computational Biology/Bioinformatics; Computational Science and Engineering; Computer Science; Data Mining and Knowledge Discovery; Deep learning; Emotion recognition; Emotions; Feature extraction; Image Processing and Computer Vision; Machine learning; Optimization; Probability and Statistics in Computer Science; Special issue on Human-in-the-loop Machine Learning and its Applications; Resnet50; VGG 19; MobileNet; Transfer learning; Inception V3; Human–computer interaction
• ISSN: 0941-0643; 1433-3058
• DOI: 10.1007/s00521-021-06012-8

One of the most significant fields in the man–machine interface is emotion recognition using facial expressions. Some of the challenges in the emotion recognition area are facial accessories, non-uniform illuminations, pose variations, etc. Emotion detection using conventional approaches having the drawback of mutual optimization of feature extraction and classification. To overcome this problem, researchers are showing more attention toward deep learning techniques. Nowadays, deep-learning approaches are playing a major role in classification tasks. This paper deals with emotion recognition by using transfer learning approaches. In this work pre-trained networks of Resnet50, vgg19, Inception V3, and Mobile Net are used. The fully connected layers of the pre-trained ConvNets are eliminated, and we add our fully connected layers that are suitable for the number of instructions in our task. Finally, the newly added layers are only trainable to update the weights. The experiment was conducted by using the CK + database and achieved an average accuracy of 96% for emotion detection problems.