Listen, Adapt, Better WER: Source-free Single-utterance Test-time Adaptation for Automatic Speech Recognition

TL;DR

This paper introduces SUTA, a novel test-time adaptation framework for automatic speech recognition that improves performance on out-of-domain and in-domain test samples using single-utterance adaptation without source data access.

Contribution

SUTA is the first to apply test-time adaptation to ASR, enabling effective single-utterance adaptation without delaying inference or requiring source data.

Findings

SUTA improves ASR accuracy on multiple out-of-domain datasets.

Single-utterance adaptation is effective without batch collection.

The method enhances in-domain test performance as well.

Abstract

Although deep learning-based end-to-end Automatic Speech Recognition (ASR) has shown remarkable performance in recent years, it suffers severe performance regression on test samples drawn from different data distributions. Test-time Adaptation (TTA), previously explored in the computer vision area, aims to adapt the model trained on source domains to yield better predictions for test samples, often out-of-domain, without accessing the source data. Here, we propose the Single-Utterance Test-time Adaptation (SUTA) framework for ASR, which is the first TTA study on ASR to our best knowledge. The single-utterance TTA is a more realistic setting that does not assume test data are sampled from identical distribution and does not delay on-demand inference due to pre-collection for the batch of adaptation data. SUTA consists of unsupervised objectives with an efficient adaptation strategy. Empirical results demonstrate that SUTA effectively improves the performance of the source ASR model evaluated on multiple out-of-domain target corpora and in-domain test samples.