RED-ACE: Robust Error Detection for ASR using Confidence Embeddings

TL;DR

This paper introduces RED-ACE, a novel approach for ASR error detection that leverages confidence embeddings from the ASR system to improve error detection accuracy and robustness.

Contribution

The paper proposes adding an ASR Confidence Embedding layer to enhance AED models by integrating confidence scores with transcribed text.

Findings

Confidence scores improve AED performance

ACE enhances robustness and effectiveness of error detection

New AED dataset on LibriSpeech with error annotations

Abstract

ASR Error Detection (AED) models aim to post-process the output of Automatic Speech Recognition (ASR) systems, in order to detect transcription errors. Modern approaches usually use text-based input, comprised solely of the ASR transcription hypothesis, disregarding additional signals from the ASR model. Instead, we propose to utilize the ASR system's word-level confidence scores for improving AED performance. Specifically, we add an ASR Confidence Embedding (ACE) layer to the AED model's encoder, allowing us to jointly encode the confidence scores and the transcribed text into a contextualized representation. Our experiments show the benefits of ASR confidence scores for AED, their complementary effect over the textual signal, as well as the effectiveness and robustness of ACE for combining these signals. To foster further research, we publish a novel AED dataset consisting of ASR outputs on the LibriSpeech corpus with annotated transcription errors.