Lattice-Based Unsupervised Test-Time Adaptation of Neural Network Acoustic Models

TL;DR

This paper introduces a lattice-based unsupervised test-time adaptation method for neural network acoustic models that improves adaptation robustness, especially in high-error transcription scenarios, by leveraging lattice information within the LF-MMI framework.

Contribution

It proposes a novel lattice-based discriminative adaptation approach that allows more parameters to be adapted without over-fitting, even with high WER initial transcriptions.

Findings

Effective adaptation on diverse transcription tasks including TED talks, MGB, and Somali.

Enables adaptation with initial WER over 50%.

Improves robustness of acoustic models to unseen test conditions.

Abstract

Acoustic model adaptation to unseen test recordings aims to reduce the mismatch between training and testing conditions. Most adaptation schemes for neural network models require the use of an initial one-best transcription for the test data, generated by an unadapted model, in order to estimate the adaptation transform. It has been found that adaptation methods using discriminative objective functions - such as cross-entropy loss - often require careful regularisation to avoid over-fitting to errors in the one-best transcriptions. In this paper we solve this problem by performing discriminative adaptation using lattices obtained from a first pass decoding, an approach that can be readily integrated into the lattice-free maximum mutual information (LF-MMI) framework. We investigate this approach on three transcription tasks of varying difficulty: TED talks, multi-genre broadcast (MGB) and a low-resource language (Somali). We find that our proposed approach enables many more parameters to be adapted without over-fitting being observed, and is successful even when the initial transcription has a WER in excess of 50%.