Cross-domain Speech Recognition with Unsupervised Character-level Distribution Matching

TL;DR

This paper introduces CMatch, a novel unsupervised character-level distribution matching method for domain adaptation in speech recognition, significantly reducing word error rates across different devices and environments.

Contribution

It proposes a new fine-grained domain adaptation technique using character-level distribution matching with pseudo labels and self-training, improving ASR performance.

Findings

Achieves 14.39% and 16.50% relative WER reduction on Libri-Adapt.

Effectively matches character distributions across domains.

Analyzes strategies for label assignment and model adaptation.

Abstract

End-to-end automatic speech recognition (ASR) can achieve promising performance with large-scale training data. However, it is known that domain mismatch between training and testing data often leads to a degradation of recognition accuracy. In this work, we focus on the unsupervised domain adaptation for ASR and propose CMatch, a Character-level distribution matching method to perform fine-grained adaptation between each character in two domains. First, to obtain labels for the features belonging to each character, we achieve frame-level label assignment using the Connectionist Temporal Classification (CTC) pseudo labels. Then, we match the character-level distributions using Maximum Mean Discrepancy. We train our algorithm using the self-training technique. Experiments on the Libri-Adapt dataset show that our proposed approach achieves 14.39% and 16.50% relative Word Error Rate (WER) reduction on both cross-device and cross-environment ASR. We also comprehensively analyze the different strategies for frame-level label assignment and Transformer adaptations.