Peeking Into The Future For Contextual Biasing

TL;DR

This paper introduces a novel attention-based encoder-decoder method that improves recognition of rare named entities in speech recognition by predicting multiple future tokens, enhancing accuracy without adding complexity.

Contribution

The proposed approach enables future token prediction for contextual biasing in AED models using only logits, reducing architectural complexity and improving named entity recognition.

Findings

Achieves up to 50.34% relative improvement in named entity WER.

Effectively incorporates candidate entity lists without additional encoders.

Reduces model complexity while enhancing rare entity recognition.

Abstract

While end-to-end (E2E) automatic speech recognition (ASR) models excel at general transcription, they struggle to recognize rare or unseen named entities (e.g., contact names, locations), which are critical for downstream applications like virtual assistants. In this paper, we propose a contextual biasing method for attention based encoder decoder (AED) models using a list of candidate named entities. Instead of predicting only the next token, we simultaneously predict multiple future tokens, enabling the model to "peek into the future" and score potential candidate entities in the entity list. Moreover, our approach leverages the multi-token prediction logits directly without requiring additional entity encoders or cross-attention layers, significantly reducing architectural complexity. Experiments on Librispeech demonstrate that our approach achieves up to 50.34% relative improvement in named entity word error rate compared to the baseline AED model.