A Neural Network Approach to Score Fusion for Emotion Recognition

Automatic facial emotion recognition has been one of the interesting research topics in the recent decades. There have been recent advances in convolutional neural networks (CNNs) which have become the state-of-the-art approaches in pattern recognition. This paper presents an effective facial emotio...

Full description

Saved in:
Bibliographic Details
Published in2018 10th Computer Science and Electronic Engineering (CEEC) pp. 180 - 185
Main Authors Basbrain, Arwa Mohammed, Gan, John Q, Sugimoto, Akihiro, Clark, Adrian
Format Conference Proceeding
LanguageEnglish
Japanese
Published IEEE 01.09.2018
Subjects
Convolutional Neural Network
Deep learning
Emotion recognition
Eyeglass Detection
Feature extraction
Glass
Multiple Classifier System
Neural Network
Neural networks
Support vector machines
Online AccessGet full text

Cover

More Information
Summary:Automatic facial emotion recognition has been one of the interesting research topics in the recent decades. There have been recent advances in convolutional neural networks (CNNs) which have become the state-of-the-art approaches in pattern recognition. This paper presents an effective facial emotion recognition system that uses convolutional neural networks (GoogleNet-CNN) for eyeglass detection and feature extraction followed by a novel score fusion model. The proposed system has three key components for improving recognition performance: 1) a highly accurate glasses detector to differentiate between images in which the human subject is wearing glasses and images in which the human subject is not wearing glasses, 2) convolutional neural networks to extract nine different sets of emotional features from faces with and without glasses, which are then classified by support vector machines (SVMs), 3) a multiple classifier system (MCS) to accomplish decision fusion by using a neural network. The USTC-NVIE (NVIE) database is used to evaluate the performance of the proposed system. Experimental results show that the eyeglass detector obtained a best overall accuracy of 99.7% and the proposed facial emotion recognition system can achieve 7-15% higher classification rates when using the eyeglass detector. By applying the neural network approach in the multiple classifier system for score fusion, the classification rates of the system increase by about 10%.
DOI:10.1109/CEEC.2018.8674191