On Unsupervised Uncertainty-Driven Speech Pseudo-Label Filtering and Model Calibration

TL;DR

This paper improves unsupervised speech domain adaptation by enhancing pseudo-label filtering through uncertainty estimation and model calibration, leading to more reliable self-training iterations.

Contribution

It introduces a theoretically grounded uncertainty-based pseudo-label filtering method and explores the impact of model calibration on self-training effectiveness.

Findings

Uncertainty-based filtering can fail under severe domain mismatch.

Calibrated models improve pseudo-label filtering reliability.

Proposed methods enhance unsupervised speech adaptation performance.

Abstract

Pseudo-label (PL) filtering forms a crucial part of Self-Training (ST) methods for unsupervised domain adaptation. Dropout-based Uncertainty-driven Self-Training (DUST) proceeds by first training a teacher model on source domain labeled data. Then, the teacher model is used to provide PLs for the unlabeled target domain data. Finally, we train a student on augmented labeled and pseudo-labeled data. The process is iterative, where the student becomes the teacher for the next DUST iteration. A crucial step that precedes the student model training in each DUST iteration is filtering out noisy PLs that could lead the student model astray. In DUST, we proposed a simple, effective, and theoretically sound PL filtering strategy based on the teacher model's uncertainty about its predictions on unlabeled speech utterances. We estimate the model's uncertainty by computing disagreement amongst multiple samples drawn from the teacher model during inference by injecting noise via dropout. In this work, we show that DUST's PL filtering, as initially used, may fail under severe source and target domain mismatch. We suggest several approaches to eliminate or alleviate this issue. Further, we bring insights from the research in neural network model calibration to DUST and show that a well-calibrated model correlates strongly with a positive outcome of the DUST PL filtering step.