CO-VADA: A Confidence-Oriented Voice Augmentation Debiasing Approach for Fair Speech Emotion Recognition

TL;DR

CO-VADA is a practical bias mitigation method for speech emotion recognition that uses voice conversion to generate diverse training samples, reducing demographic bias without changing model architecture or needing demographic labels.

Contribution

It introduces a scalable, model-agnostic approach that leverages voice conversion to create bias-diverse training data, enhancing fairness in SER systems.

Findings

Improves fairness across demographic groups in SER.

Compatible with various SER models and voice conversion tools.

Does not require demographic annotations or model modifications.

Abstract

Bias in speech emotion recognition (SER) systems often stems from spurious correlations between speaker characteristics and emotional labels, leading to unfair predictions across demographic groups. Many existing debiasing methods require model-specific changes or demographic annotations, limiting their practical use. We present CO-VADA, a Confidence-Oriented Voice Augmentation Debiasing Approach that mitigates bias without modifying model architecture or relying on demographic information. CO-VADA identifies training samples that reflect bias patterns present in the training data and then applies voice conversion to alter irrelevant attributes and generate samples. These augmented samples introduce speaker variations that differ from dominant patterns in the data, guiding the model to focus more on emotion-relevant features. Our framework is compatible with various SER models and voice conversion tools, making it a scalable and practical solution for improving fairness in SER systems.