Learning More with Less: Self-Supervised Approaches for Low-Resource Speech Emotion Recognition

TL;DR

This paper explores self-supervised learning methods like contrastive learning and BYOL to improve speech emotion recognition in low-resource languages, achieving significant performance gains and providing insights into cross-lingual generalization challenges.

Contribution

It introduces self-supervised approaches for low-resource SER, demonstrating their effectiveness and analyzing factors influencing cross-lingual performance.

Findings

F1 score improvements of 10.6% in Urdu

F1 score improvements of 15.2% in German

F1 score improvements of 13.9% in Bangla

Abstract

Speech Emotion Recognition (SER) has seen significant progress with deep learning, yet remains challenging for Low-Resource Languages (LRLs) due to the scarcity of annotated data. In this work, we explore unsupervised learning to improve SER in low-resource settings. Specifically, we investigate contrastive learning (CL) and Bootstrap Your Own Latent (BYOL) as self-supervised approaches to enhance cross-lingual generalization. Our methods achieve notable F1 score improvements of 10.6% in Urdu, 15.2% in German, and 13.9% in Bangla, demonstrating their effectiveness in LRLs. Additionally, we analyze model behavior to provide insights on key factors influencing performance across languages, and also highlighting challenges in low-resource SER. This work provides a foundation for developing more inclusive, explainable, and robust emotion recognition systems for underrepresented languages.