BR-ASR: Efficient and Scalable Bias Retrieval Framework for Contextual Biasing ASR in Speech LLM

TL;DR

BR-ASR introduces a scalable bias retrieval framework for speech recognition that effectively handles large bias lists, improving accuracy and efficiency without fine-tuning, and achieves state-of-the-art results on LibriSpeech.

Contribution

It proposes a novel contrastive learning and curriculum learning approach for large-scale bias retrieval in ASR, enabling seamless integration and high scalability.

Findings

Achieves SOTA biased word error rates of 2.8%/7.1%.

Maintains high accuracy with 200k bias entries, only minor WER degradation.

Operates with 20ms latency per query, demonstrating real-time capability.

Abstract

While speech large language models (SpeechLLMs) have advanced standard automatic speech recognition (ASR), contextual biasing for named entities and rare words remains challenging, especially at scale. To address this, we propose BR-ASR: a Bias Retrieval framework for large-scale contextual biasing (up to 200k entries) via two innovations: (1) speech-and-bias contrastive learning to retrieve semantically relevant candidates; (2) dynamic curriculum learning that mitigates homophone confusion which negatively impacts the final performance. The is a general framework that allows seamless integration of the retrieved candidates into diverse ASR systems without fine-tuning. Experiments on LibriSpeech test-clean/-other achieve state-of-the-art (SOTA) biased word error rates (B-WER) of 2.8%/7.1% with 2000 bias words, delivering 45% relative improvement over prior methods. BR-ASR also demonstrates high scalability: when expanding the bias list to 200k where traditional methods generally fail, it induces only 0.3 / 2.9% absolute WER / B-WER degradation with a 99.99% pruning rate and only 20ms latency per query on test-other.