Anchored Speech Recognition with Neural Transducers

TL;DR

This paper enhances neural transducer speech recognition by incorporating anchor segments to improve robustness against background speech, achieving significant error rate reductions in challenging acoustic conditions.

Contribution

It introduces a novel anchored speech recognition approach using auxiliary networks and biasing techniques to improve neural transducer robustness in noisy environments.

Findings

Achieved 19.6% relative WER reduction over baseline.

Effective disentanglement of lexical content from speaking style.

Improved robustness across various SNR and overlap conditions.

Abstract

Neural transducers have achieved human level performance on standard speech recognition benchmarks. However, their performance significantly degrades in the presence of cross-talk, especially when the primary speaker has a low signal-to-noise ratio. Anchored speech recognition refers to a class of methods that use information from an anchor segment (e.g., wake-words) to recognize device-directed speech while ignoring interfering background speech. In this paper, we investigate anchored speech recognition to make neural transducers robust to background speech. We extract context information from the anchor segment with a tiny auxiliary network, and use encoder biasing and joiner gating to guide the transducer towards the target speech. Moreover, to improve the robustness of context embedding extraction, we propose auxiliary training objectives to disentangle lexical content from speaking style. We evaluate our methods on synthetic LibriSpeech-based mixtures comprising several SNR and overlap conditions; they improve relative word error rates by 19.6% over a strong baseline, when averaged over all conditions.