End-to-End Speech Emotion Recognition: Challenges of Real-Life Emergency Call Centers Data Recordings

• Published in: International Conference on Affective Computing and Intelligent Interaction and workshops pp. 1 - 8
• Main Authors: Deschamps-Berger, Theo, Lamel, Lori, Devillers, Laurence
• Format: Conference Proceeding
• Language: English
• Published: IEEE 28.09.2021
• Subjects: Affective computing; call center; complex emotions; Computer architecture; Deep learning; Diversity reception; emotion detection; Emotion recognition; end-to-end deep learning architecture; real-life database; Speech recognition
• ISSN: 2156-8111
• DOI: 10.1109/ACII52823.2021.9597419

Recognizing a speaker's emotion from their speech can be a key element in emergency call centers. End-to-end deep learning systems for speech emotion recognition now achieve equivalent or even better results than conventional machine learning approaches. In this paper, in order to validate the performance of our neural network architecture for emotion recognition from speech, we first trained and tested it on the widely used corpus accessible by the community, IEMOCAP. We then used the same architecture with the real life corpus, CEMO, comprised of 440 dialogs (2h16m) from 485 speakers. The most frequent emotions expressed by callers in these real-life emergency dialogues are fear, anger and positive emotions such as relief. In the IEMOCAP general topic conversations, the most frequent emotions are sadness, anger and happiness. Using the same end-to-end deep learning architecture, an Unweighted Accuracy Recall (UA) of 63% is obtained on IEMOCAP and a UA of 45.6% on CEMO, each with 4 classes. Using only 2 classes (Anger, Neutral), the results for CEMO are 76.9% UA compared to 81.1% UA for IEMOCAP. We expect that these encouraging results with CEMO can be improved by combining the audio channel with the linguistic channel. Real-life emotions are clearly more complex than acted ones, mainly due to the large diversity of emotional expressions of speakers.