# On Modeling ASR Word Confidence

## TL;DR

This paper introduces a novel approach for computing and calibrating ASR word confidence scores using a Heterogeneous Word Confusion Network and lattice RNNs, enhancing accuracy and comparability across models.

## Contribution

It proposes a new HWCN-based confidence modeling method and a score calibration technique, addressing flaws in existing methods and improving downstream application performance.

## Key findings

- HWCN-based confidence scores outperform traditional methods.
- Calibration improves recognizer combination reliability.
- Best confidence sequence exceeds default 1-best accuracy.

## Abstract

We present a new method for computing ASR word confidences that effectively mitigates the effect of ASR errors for diverse downstream applications, improves the word error rate of the 1-best result, and allows better comparison of scores across different models. We propose 1) a new method for modeling word confidence using a Heterogeneous Word Confusion Network (HWCN) that addresses some key flaws in conventional Word Confusion Networks, and 2) a new score calibration method for facilitating direct comparison of scores from different models. Using a bidirectional lattice recurrent neural network to compute the confidence scores of each word in the HWCN, we show that the word sequence with the best overall confidence is more accurate than the default 1-best result of the recognizer, and that the calibration method can substantially improve the reliability of recognizer combination.