Self-Train Before You Transcribe

TL;DR

This paper proposes a test-time adaptation method for speech recognition that applies noisy student teacher training directly on test recordings, significantly improving performance under domain mismatch without needing separate adaptation data.

Contribution

It introduces a novel test-time self-training approach for speech recognition that enhances domain adaptation by leveraging test recordings themselves.

Findings

Achieves up to 32.2% relative performance gains

Outperforms traditional self-training with separate adaptation data

Effective across various in-domain and out-of-domain datasets

Abstract

When there is a mismatch between the training and test domains, current speech recognition systems show significant performance degradation. Self-training methods, such as noisy student teacher training, can help address this and enable the adaptation of models under such domain shifts. However, self-training typically requires a collection of unlabelled target domain data. For settings where this is not practical, we investigate the benefit of performing noisy student teacher training on recordings in the test set as a test-time adaptation approach. Similarly to the dynamic evaluation approach in language modelling, this enables the transfer of information across utterance boundaries and functions as a method of domain adaptation. A range of in-domain and out-of-domain datasets are used for experiments demonstrating large relative gains of up to 32.2%. Interestingly, our method showed larger gains than the typical self-training setup that utilises separate adaptation data.