Fast Entropy-Based Methods of Word-Level Confidence Estimation for End-To-End Automatic Speech Recognition

TL;DR

This paper introduces fast, entropy-based confidence estimation methods for end-to-end speech recognition models that outperform traditional probability-based methods in detecting errors.

Contribution

It proposes new non-trainable entropy-based confidence measures that are computationally efficient, more adjustable, and more effective at error detection than existing methods.

Findings

Entropy-based confidence measures outperform probability-based methods in error detection.

Methods are computationally similar to traditional approaches.

Significant improvements in incorrect word detection on LibriSpeech.

Abstract

This paper presents a class of new fast non-trainable entropy-based confidence estimation methods for automatic speech recognition. We show how per-frame entropy values can be normalized and aggregated to obtain a confidence measure per unit and per word for Connectionist Temporal Classification (CTC) and Recurrent Neural Network Transducer (RNN-T) models. Proposed methods have similar computational complexity to the traditional method based on the maximum per-frame probability, but they are more adjustable, have a wider effective threshold range, and better push apart the confidence distributions of correct and incorrect words. We evaluate the proposed confidence measures on LibriSpeech test sets, and show that they are up to 2 and 4 times better than confidence estimation based on the maximum per-frame probability at detecting incorrect words for Conformer-CTC and Conformer-RNN-T models, respectively.