# Optimizing expected word error rate via sampling for speech recognition

## TL;DR

This paper introduces a sampling-based method to optimize expected word error rate (WER) directly during speech recognition training, leading to significant improvements over traditional sMBR training.

## Contribution

It presents a novel Monte Carlo sampling approach to optimize expected WER, overcoming limitations of previous exact computation methods.

## Key findings

- Achieved 5% relative WER reduction on Google Home query recognition task.
- Demonstrated effective optimization of expected WER using sampling during training.
- Outperformed standard sMBR training in experimental evaluations.

## Abstract

State-level minimum Bayes risk (sMBR) training has become the de facto standard for sequence-level training of speech recognition acoustic models. It has an elegant formulation using the expectation semiring, and gives large improvements in word error rate (WER) over models trained solely using cross-entropy (CE) or connectionist temporal classification (CTC). sMBR training optimizes the expected number of frames at which the reference and hypothesized acoustic states differ. It may be preferable to optimize the expected WER, but WER does not interact well with the expectation semiring, and previous approaches based on computing expected WER exactly involve expanding the lattices used during training. In this paper we show how to perform optimization of the expected WER by sampling paths from the lattices used during conventional sMBR training. The gradient of the expected WER is itself an expectation, and so may be approximated using Monte Carlo sampling. We show experimentally that optimizing WER during acoustic model training gives 5% relative improvement in WER over a well-tuned sMBR baseline on a 2-channel query recognition task (Google Home).