Exploring Fusion Strategies in Deep Multimodal Affect Prediction

• Published in: Image Analysis and Processing - ICIAP 2022 Vol. 13232; pp. 730 - 741
• Main Authors: Patania, Sabrina, D’Amelio, Alessandro, Lanzarotti, Raffaella
• Format: Book Chapter
• Language: English
• Published: Switzerland Springer International Publishing AG 2022; Springer International Publishing
• Series: Lecture Notes in Computer Science
• Subjects: Deep learning; Multimodal emotion recognition; Multimodal fusion

In this work, we explore the effectiveness of multimodal models for estimating the emotional state expressed continuously in the Valence/Arousal space. We consider four modalities typically adopted for the emotion recognition, namely audio (voice), video (face expression), electrocardiogram (ECG), a...