End-to-End Speech Emotion Recognition: Challenges of Real-Life Emergency Call Centers Data Recordings

TL;DR

This study evaluates an end-to-end deep learning model for speech emotion recognition in emergency call centers, comparing performance on a standard dataset and real-life recordings, highlighting challenges of complex emotions in real scenarios.

Contribution

It applies and tests a neural network architecture for speech emotion recognition on both a standard and a real-life emergency call dataset, revealing real-world challenges.

Findings

63% UA on IEMOCAP with 4 emotions

45.6% UA on CEMO with 4 emotions

76.9% UA with 2 emotions on CEMO

Abstract

Recognizing a speaker's emotion from their speech can be a key element in emergency call centers. End-to-end deep learning systems for speech emotion recognition now achieve equivalent or even better results than conventional machine learning approaches. In this paper, in order to validate the performance of our neural network architecture for emotion recognition from speech, we first trained and tested it on the widely used corpus accessible by the community, IEMOCAP. We then used the same architecture as the real life corpus, CEMO, composed of 440 dialogs (2h16m) from 485 speakers. The most frequent emotions expressed by callers in these real life emergency dialogues are fear, anger and positive emotions such as relief. In the IEMOCAP general topic conversations, the most frequent emotions are sadness, anger and happiness. Using the same end-to-end deep learning architecture, an Unweighted Accuracy Recall (UA) of 63% is obtained on IEMOCAP and a UA of 45.6% on CEMO, each with 4 classes. Using only 2 classes (Anger, Neutral), the results for CEMO are 76.9% UA compared to 81.1% UA for IEMOCAP. We expect that these encouraging results with CEMO can be improved by combining the audio channel with the linguistic channel. Real-life emotions are clearly more complex than acted ones, mainly due to the large diversity of emotional expressions of speakers. Index Terms-emotion detection, end-to-end deep learning architecture, call center, real-life database, complex emotions.