An Effective Context-Balanced Adaptation Approach for Long-Tailed Speech Recognition

TL;DR

This paper proposes a context-balanced adaptation method for long-tailed speech recognition, improving recognition of rare and zero-shot words by addressing data imbalance issues in contextual modeling.

Contribution

It introduces a simple context-balanced learning objective and explores the impact of context list composition, significantly enhancing rare word recognition in E2E ASR models.

Findings

Using all vocabulary words as context improves performance.

The balanced objective reduces CER by up to 1.21%.

Zero-shot word error rate decreases by 9.44%.

Abstract

End-to-end (E2E) automatic speech recognition (ASR) models have become standard practice for various commercial applications. However, in real-world scenarios, the long-tailed nature of word distribution often leads E2E ASR models to perform well on common words but fall short in recognizing uncommon ones. Recently, the notion of a contextual adapter (CA) was proposed to infuse external knowledge represented by a context word list into E2E ASR models. Although CA can improve recognition performance on rare words, two crucial data imbalance problems remain. First, when using low-frequency words as context words during training, since these words rarely occur in the utterance, CA becomes prone to overfit on attending to the <no-context> token due to higher-frequency words not being present in the context list. Second, the long-tailed distribution within the context list itself still causes the model to perform poorly on low-frequency context words. In light of this, we explore in-depth the impact of altering the context list to have words with different frequency distributions on model performance, and meanwhile extend CA with a simple yet effective context-balanced learning objective. A series of experiments conducted on the AISHELL-1 benchmark dataset suggests that using all vocabulary words from the training corpus as the context list and pairing them with our balanced objective yields the best performance, demonstrating a significant reduction in character error rate (CER) by up to 1.21% and a more pronounced 9.44% reduction in the error rate of zero-shot words.