EMOVOME: A Dataset for Emotion Recognition in Spontaneous Real-Life Speech

TL;DR

This paper introduces EMOVOME, a new dataset of spontaneous real-life speech emotions from Spanish speakers, and evaluates state-of-the-art models showing significant improvements over baselines and highlighting challenges in real-world emotion recognition.

Contribution

The paper presents EMOVOME, a publicly available dataset of spontaneous speech emotions in real-life conversations, and benchmarks SER models demonstrating the gap between controlled and real-world data.

Findings

Pre-trained UniSpeech-SAT-Large achieved 61.64% UA for valence prediction.

EMOVOME outperformed baseline models by approximately 10%.

Combining expert and non-expert annotations improved fairness and results.

Abstract

Spontaneous datasets for Speech Emotion Recognition (SER) are scarce and frequently derived from laboratory environments or staged scenarios, such as TV shows, limiting their application in real-world contexts. We developed and publicly released the Emotional Voice Messages (EMOVOME) dataset, including 999 voice messages from real conversations of 100 Spanish speakers on a messaging app, labeled in continuous and discrete emotions by expert and non-expert annotators. We evaluated speaker-independent SER models using acoustic features as baseline and transformer-based models. We compared the results with reference datasets including acted and elicited speech, and analyzed the influence of annotators and gender fairness. The pre-trained UniSpeech-SAT-Large model achieved the highest results, 61.64% and 55.57% Unweighted Accuracy (UA) for 3-class valence and arousal prediction respectively on EMOVOME, a 10% improvement over baseline models. For the emotion categories, 42.58% UA was obtained. EMOVOME performed lower than the acted RAVDESS dataset. The elicited IEMOCAP dataset also outperformed EMOVOME in predicting emotion categories, while similar results were obtained in valence and arousal. EMOVOME outcomes varied with annotator labels, showing better results and fairness when combining expert and non-expert annotations. This study highlights the gap between controlled and real-life scenarios, supporting further advancements in recognizing genuine emotions.