Audio-attention discriminative language model for ASR rescoring

TL;DR

This paper introduces an attention-based discriminative language model that effectively rescoring ASR outputs, significantly improving accuracy with less training data compared to end-to-end models.

Contribution

It presents a novel approach combining end-to-end and conventional systems through an attention-based rescoring model that simplifies training and enhances ASR performance.

Findings

Achieves 8% WER reduction in ASR rescoring

Requires less training data than first-pass models

Effectively combines end-to-end and traditional ASR benefits

Abstract

End-to-end approaches for automatic speech recognition (ASR) benefit from directly modeling the probability of the word sequence given the input audio stream in a single neural network. However, compared to conventional ASR systems, these models typically require more data to achieve comparable results. Well-known model adaptation techniques, to account for domain and style adaptation, are not easily applicable to end-to-end systems. Conventional HMM-based systems, on the other hand, have been optimized for various production environments and use cases. In this work, we propose to combine the benefits of end-to-end approaches with a conventional system using an attention-based discriminative language model that learns to rescore the output of a first-pass ASR system. We show that learning to rescore a list of potential ASR outputs is much simpler than learning to generate the hypothesis. The proposed model results in 8% improvement in word error rate even when the amount of training data is a fraction of data used for training the first-pass system.