Transliterated Zero-Shot Domain Adaptation for Automatic Speech Recognition

TL;DR

This paper introduces a transliteration-based zero-shot domain adaptation method for automatic speech recognition, enabling models to adapt to new domains in unseen languages without target domain data.

Contribution

It proposes transliterated ZSDA, preserving pre-trained knowledge during fine-tuning, and demonstrates significant improvements over baseline methods in cross-lingual speech recognition.

Findings

Reduces word error rate by 9.2% compared to baseline

Outperforms self-supervised ZSDA methods

Matches supervised ZSDA performance

Abstract

The performance of automatic speech recognition models often degenerates on domains not covered by the training data. Domain adaptation can address this issue, assuming the availability of the target domain data in the target language. However, such assumption does not stand in many real-world applications. To make domain adaptation more applicable, we address the problem of zero-shot domain adaptation (ZSDA), where target domain data is unavailable in the target language. Instead, we transfer the target domain knowledge from another source language where the target domain data is more accessible. To do that, we first perform cross-lingual pre-training (XLPT) to share domain knowledge across languages, then use target language fine-tuning to build the final model. One challenge in this practice is that the pre-trained knowledge can be forgotten during fine-tuning, resulting in sub-optimal adaptation performance. To address this issue, we propose transliterated ZSDA to achieve consistent pre-training and fine-tuning labels, leading to maximum preservation of the pre-trained knowledge. Experimental results show that transliterated ZSDA relatively decreases the word error rate by 9.2% compared with a wav2vec 2.0 baseline. Moreover, transliterated ZSDA consistently outperforms self-supervised ZSDA and performs on par with supervised ZSDA, proving the superiority of transliteration-based pre-training labels.