Multimodal Emotion Recognition Using Different Fusion Techniques

Human beings have the ability to understand and visualize various emotions on a daily basis. This could be done by noticing various features such as facial muscle movements, speech, hand gestures, etc. The automated emotion recognition is an important issue and has also been a lively research topic...

Full description

Saved in:
Bibliographic Details
Published in2021 Seventh International conference on Bio Signals, Images, and Instrumentation (ICBSII) pp. 1 - 6
Main Authors Subramanian, Gokul, Cholendiran, Niranjan, Prathyusha, Kotapati, Balasubramanain, Noviya, Aravinth, J
Format Conference Proceeding
LanguageEnglish
Published IEEE 25.03.2021
Subjects
Autoencoder
Convolutional neural network (CNN)
Deep learning
Electroencephalogram (EEG)
Emotion recognition
Face recognition
Feature fusion
Instruments
Multimodal
Neural networks
Speech recognition
Support vector machines
Time series analysis
Online AccessGet full text

Cover

More Information
Summary:Human beings have the ability to understand and visualize various emotions on a daily basis. This could be done by noticing various features such as facial muscle movements, speech, hand gestures, etc. The automated emotion recognition is an important issue and has also been a lively research topic for the modern time. At the moment, several research workers have taken part in inheriting two or more unimodals for better understanding. This paper shows an approach for emotion recognition that uses three modalities: facial images, audio signals, and electroencephalogram (EEG) signals from FER and Ck+, RAVDESS and SEED-IV datasets respectively. Finally, various fusion techniques were approached and each of these fusion methods gave different results. The maximum accuracy of 71.24% was obtained with help of an autoencoder approach when combined with SVM classifier.
DOI:10.1109/ICBSII51839.2021.9445146