Detecting Affect States Using VGG16, ResNet50 and SE-ResNet50 Networks

• Published in: SN computer science Vol. 1; no. 2; p. 79
• Main Authors: Theckedath, Dhananjay, Sedamkar, R. R.
• Format: Journal Article
• Language: English
• Published: Singapore Springer Nature Singapore 01.03.2020; Springer Nature B.V
• Subjects: Accuracy; Advances in Computational Intelligence; Affect (Psychology); Algorithms; Architecture; Artificial neural networks; Classification; Computer Imaging; Computer Science; Computer Systems Organization and Communication Networks; Data Structures and Information Theory; Datasets; Deep learning; Human-computer interface; Information Systems and Communication Service; Neural networks; Original Research; Paradigms and Applications; Pattern Recognition and Graphics; Software Engineering/Programming and Operating Systems; Vision; Affect states; Transfer learning; Convolutional neural network
• ISSN: 2662-995X; 2661-8907
• DOI: 10.1007/s42979-020-0114-9

Affect detection is a key component in developing intelligent human computer interface systems. State-of-the-art affect detection systems assume the availability of full un-occluded face images. This work uses convolutional neural networks with transfer learning to detect 7 basic affect states, viz. Angry, Contempt, Disgust, Fear, Happy and Sad. The paper compares three pre-trained networks, viz. VGG16, ResNet50 and a SE-ResNet50, in which a new architectural block of squeeze and excitation has been integrated with ResNet50. Modified VGG-16, ResNet50 and SE-ResNet50 networks are trained on images from the dataset, and the results are compared. We have been able to achieve validation accuracies of 96.8%, 99.47%, and 97.34% for VGG16, ResNet50 and SE-ResNet50, respectively. Apart from accuracy, the other performance matrices used in this work are precision and recall. Our evaluation, based on these performance matrices, shows that accurate affect detection is obtained from all the three networks with Resnet50 being the most accurate.