Context Biasing for Pronunciation-Orthography Mismatch in Automatic Speech Recognition

TL;DR

This paper introduces a context biasing method for speech recognition that allows on-the-fly corrections to handle pronunciation-orthography mismatches, significantly improving recognition accuracy for challenging words.

Contribution

The proposed method enables real-time correction of substitution errors, enhancing recognition of unseen words without degrading overall system performance.

Findings

Achieved 22-34% relative improvement in biased word error rate

Effective handling of pronunciation-orthography mismatches

Maintains overall recognition performance

Abstract

Neural sequence-to-sequence systems deliver state-of-the-art performance for automatic speech recognition. When using appropriate modeling units, e.g., byte-pair encoding, these systems are in principle open vocabulary systems. In practice, however, they often fail to recognize words not seen during training, e.g., named entities, acronyms, or domain-specific special words. To address this problem, many context biasing methods have been proposed; however, these methods may still struggle when they are unable to relate audio and corresponding text, e.g., in case of a pronunciation-orthography mismatch. We propose a method where corrections of substitution errors can be used to improve the recognition accuracy of such challenging words. Users can add corrections on the fly during inference. We show that with this method we get a relative improvement in biased word error rate between 22% and 34% compared to a text-based replacement method, while maintaining the overall performance.