Delay-penalized transducer for low-latency streaming ASR

TL;DR

This paper introduces a simple delay-penalized transducer method for streaming ASR that effectively reduces symbol delay while maintaining recognition accuracy, offering a more justified approach than previous methods.

Contribution

It proposes a novel delay penalty technique for transducer models that balances delay and accuracy without relying on external alignments.

Findings

Significantly reduces symbol delay in streaming ASR models.

Achieves similar delay-accuracy trade-off to FastEmit with better theoretical justification.

Applicable to both Conformer and LSTM models.

Abstract

In streaming automatic speech recognition (ASR), it is desirable to reduce latency as much as possible while having minimum impact on recognition accuracy. Although a few existing methods are able to achieve this goal, they are difficult to implement due to their dependency on external alignments. In this paper, we propose a simple way to penalize symbol delay in transducer model, so that we can balance the trade-off between symbol delay and accuracy for streaming models without external alignments. Specifically, our method adds a small constant times (T/2 - t), where T is the number of frames and t is the current frame, to all the non-blank log-probabilities (after normalization) that are fed into the two dimensional transducer recursion. For both streaming Conformer models and unidirectional long short-term memory (LSTM) models, experimental results show that it can significantly reduce the symbol delay with an acceptable performance degradation. Our method achieves similar delay-accuracy trade-off to the previously published FastEmit, but we believe our method is preferable because it has a better justification: it is equivalent to penalizing the average symbol delay. Our work is open-sourced and publicly available (https://github.com/k2-fsa/k2).