No Pitch Left Behind: Addressing Gender Unbalance in Automatic Speech Recognition through Pitch Manipulation

TL;DR

This paper introduces a data augmentation method manipulating pitch features to mitigate gender bias in end-to-end speech recognition, significantly improving recognition accuracy for female speakers.

Contribution

It presents a novel pitch manipulation data augmentation technique specifically designed to address gender imbalance in end-to-end ASR systems.

Findings

Up to 9.87% relative WER reduction for female speakers.

Greater improvements for under-represented pitch ranges.

Enhanced gender fairness in recognition accuracy.

Abstract

Automatic speech recognition (ASR) systems are known to be sensitive to the sociolinguistic variability of speech data, in which gender plays a crucial role. This can result in disparities in recognition accuracy between male and female speakers, primarily due to the under-representation of the latter group in the training data. While in the context of hybrid ASR models several solutions have been proposed, the gender bias issue has not been explicitly addressed in end-to-end neural architectures. To fill this gap, we propose a data augmentation technique that manipulates the fundamental frequency (f0) and formants. This technique reduces the data unbalance among genders by simulating voices of the under-represented female speakers and increases the variability within each gender group. Experiments on spontaneous English speech show that our technique yields a relative WER improvement up to 9.87% for utterances by female speakers, with larger gains for the least-represented f0 ranges.