EMO-Debias: Benchmarking Gender Debiasing Techniques in Multi-Label Speech Emotion Recognition

TL;DR

This paper benchmarks 13 gender debiasing methods in multi-label speech emotion recognition, analyzing their effectiveness and robustness across datasets, and providing insights into fairness-accuracy trade-offs.

Contribution

It introduces EMO-Debias, a comprehensive benchmark for evaluating gender debiasing techniques in multi-label SER, covering diverse methods and datasets.

Findings

Certain methods consistently reduce gender bias without harming accuracy

Dataset distribution significantly impacts debiasing effectiveness

Trade-offs exist between fairness improvements and overall performance

Abstract

Speech emotion recognition (SER) systems often exhibit gender bias. However, the effectiveness and robustness of existing debiasing methods in such multi-label scenarios remain underexplored. To address this gap, we present EMO-Debias, a large-scale comparison of 13 debiasing methods applied to multi-label SER. Our study encompasses techniques from pre-processing, regularization, adversarial learning, biased learners, and distributionally robust optimization. Experiments conducted on acted and naturalistic emotion datasets, using WavLM and XLSR representations, evaluate each method under conditions of gender imbalance. Our analysis quantifies the trade-offs between fairness and accuracy, identifying which approaches consistently reduce gender performance gaps without compromising overall model performance. The findings provide actionable insights for selecting effective debiasing strategies and highlight the impact of dataset distributions.