Hybrid Decoding: Rapid Pass and Selective Detailed Correction for Sequence Models

TL;DR

This paper introduces Hybrid Decoding, a method combining a fast auxiliary decoder with a traditional transformer decoder to speed up inference and reduce repetition errors in speech recognition, achieving faster and more accurate results.

Contribution

The paper proposes a novel hybrid decoding approach that extends transformer models with a lightweight fast decoder for rapid inference and selective correction, improving speed and robustness.

Findings

More than doubled inference speed on speech recognition benchmarks.

Achieved comparable or better word error rates than baseline models.

Enhanced robustness against repetitive errors in recognition outputs.

Abstract

Recently, Transformer-based encoder-decoder models have demonstrated strong performance in multilingual speech recognition. However, the decoder's autoregressive nature and large size introduce significant bottlenecks during inference. Additionally, although rare, repetition can occur and negatively affect recognition accuracy. To tackle these challenges, we propose a novel Hybrid Decoding approach that both accelerates inference and alleviates the issue of repetition. Our method extends the transformer encoder-decoder architecture by attaching a lightweight, fast decoder to the pretrained encoder. During inference, the fast decoder rapidly generates an output, which is then verified and, if necessary, selectively corrected by the Transformer decoder. This results in faster decoding and improved robustness against repetitive errors. Experiments on the LibriSpeech and GigaSpeech test sets indicate that, with fine-tuning limited to the added decoder, our method achieves word error rates comparable to or better than the baseline, while more than doubling the inference speed.