To train or not to train adversarially: A study of bias mitigation strategies for speaker recognition

TL;DR

This paper evaluates biases in speaker recognition systems and proposes adversarial and multi-task learning methods to improve fairness, analyzing the trade-offs between fairness and system utility.

Contribution

It systematically assesses gender bias across system operating points and introduces novel adversarial and multi-task learning techniques for bias mitigation in speaker recognition.

Findings

Adversarial training improves fairness but reduces utility.

Multi-task learning enhances fairness while maintaining utility.

Bias mitigation strategies vary in effectiveness depending on system settings.

Abstract

Speaker recognition is increasingly used in several everyday applications including smart speakers, customer care centers and other speech-driven analytics. It is crucial to accurately evaluate and mitigate biases present in machine learning (ML) based speech technologies, such as speaker recognition, to ensure their inclusive adoption. ML fairness studies with respect to various demographic factors in modern speaker recognition systems are lagging compared to other human-centered applications such as face recognition. Existing studies on fairness in speaker recognition systems are largely limited to evaluating biases at specific operating points of the systems, which can lead to false expectations of fairness. Moreover, there are only a handful of bias mitigation strategies developed for speaker recognition systems. In this paper, we systematically evaluate the biases present in speaker recognition systems with respect to gender across a range of system operating points. We also propose adversarial and multi-task learning techniques to improve the fairness of these systems. We show through quantitative and qualitative evaluations that the proposed methods improve the fairness of ASV systems over baseline methods trained using data balancing techniques. We also present a fairness-utility trade-off analysis to jointly examine fairness and the overall system performance. We show that although systems trained using adversarial techniques improve fairness, they are prone to reduced utility. On the other hand, multi-task methods can improve the fairness while retaining the utility. These findings can inform the choice of bias mitigation strategies in the field of speaker recognition.