DYNAC: Dynamic Vocabulary based Non-Autoregressive Contextualization for Speech Recognition

TL;DR

DYNAC introduces a self-conditioned CTC approach that integrates dynamic vocabulary into speech recognition, significantly improving inference speed while maintaining accuracy on LibriSpeech.

Contribution

It proposes a novel self-conditioned CTC method that effectively incorporates dynamic vocabulary into non-autoregressive speech recognition models.

Findings

Reduces real-time factor (RTF) by 81%

Degrades word error rate by only 0.1 points

Enhances speed without sacrificing accuracy

Abstract

Contextual biasing (CB) improves automatic speech recognition for rare and unseen phrases. Recent studies have introduced dynamic vocabulary, which represents context phrases as expandable tokens in autoregressive (AR) models. This method improves CB accuracy but with slow inference speed. While dynamic vocabulary can be applied to non-autoregressive (NAR) models, such as connectionist temporal classification (CTC), the conditional independence assumption fails to capture dependencies between static and dynamic tokens. This paper proposes DYNAC (Dynamic Vocabulary-based NAR Contextualization), a self-conditioned CTC method that integrates dynamic vocabulary into intermediate layers. Conditioning the encoder on dynamic vocabulary, DYNAC effectively captures dependencies between static and dynamic tokens while reducing the real-time factor (RTF). Experimental results show that DYNAC reduces RTF by 81% with a 0.1-point degradation in word error rate on the LibriSpeech 960 test-clean set.