FT-Boosted SV: Towards Noise Robust Speaker Verification for English Speaking Classroom Environments

TL;DR

This paper proposes finetuning speaker verification models with augmented children's datasets to improve noise robustness in classroom environments, significantly reducing error rates.

Contribution

The study demonstrates that finetuning with augmented children's datasets effectively enhances speaker verification accuracy in noisy classroom settings.

Findings

ECAPA-TDNN EER reduced by 5% on MPT dataset

x-vector EER reduced by 8% on NCTE dataset

Finetuning improves noise robustness for classroom speaker verification

Abstract

Creating Speaker Verification (SV) systems for classroom settings that are robust to classroom noises such as babble noise is crucial for the development of AI tools that assist educational environments. In this work, we study the efficacy of finetuning with augmented children datasets to adapt the x-vector and ECAPA-TDNN to classroom environments. We demonstrate that finetuning with augmented children's datasets is powerful in that regard and reduces the Equal Error Rate (EER) of x-vector and ECAPA-TDNN models for both classroom datasets and children speech datasets. Notably, this method reduces EER of the ECAPA-TDNN model on average by half (a 5 % improvement) for classrooms in the MPT dataset compared to the ECAPA-TDNN baseline model. The x-vector model shows an 8 % average improvement for classrooms in the NCTE dataset compared to its baseline.