A Multi-modal Approach for Emotion Recognition of TV Drama Characters Using Image and Text

Research on facial emotion recognition has long been popular for various purposes. This paper investigates the recognition of the character emotions, to assist in understanding the story. The goal of this research is to classify the facial images of the characters in the Korean TV series 'Misae...

Full description

Saved in:
Bibliographic Details
Published in2020 IEEE International Conference on Big Data and Smart Computing (BigComp) pp. 420 - 424
Main Authors Lee, Jung-Hoon, Kim, Hyun-Ju, Cheong, Yun-Gyung
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.02.2020
Subjects
Asian-face
Character recognition
CNN
contextualized-embedding
Emotion recognition
Face recognition
face-emotion-classification
Image coding
Machine learning
multi-modal
Videos
Online AccessGet full text

Cover

More Information
Summary:Research on facial emotion recognition has long been popular for various purposes. This paper investigates the recognition of the character emotions, to assist in understanding the story. The goal of this research is to classify the facial images of the characters in the Korean TV series 'Misaeng: The Incomplete'1 into 7 emotions: Angry, Disgust, Fear, Happy, Neutral, Sad, and Surprise. We built a multi-modal deep learning model which utilizes facial images as well as textual information describing the situations, to classify the facial images. Our experiments indicate that employing multi-modality enhances the performance of facial emotion recognition of story characters.We concludes with discussions and future work.
ISSN:2375-9356
DOI:10.1109/BigComp48618.2020.00-37