Transformer Transducer: One Model Unifying Streaming and Non-streaming Speech Recognition

TL;DR

This paper introduces a Transformer-Transducer model that unifies streaming and non-streaming speech recognition, allowing flexible latency-accuracy trade-offs and achieving significant accuracy improvements with minimal additional latency.

Contribution

The paper proposes a novel Transformer-Transducer architecture with variable right context layers, enabling a single model to perform both streaming and non-streaming speech recognition.

Findings

Achieves 20% relative accuracy improvement on voice-search task.

Allows dynamic adjustment of context length for latency-accuracy trade-offs.

Optimizations enable faster inference in both streaming and non-streaming modes.

Abstract

In this paper we present a Transformer-Transducer model architecture and a training technique to unify streaming and non-streaming speech recognition models into one model. The model is composed of a stack of transformer layers for audio encoding with no lookahead or right context and an additional stack of transformer layers on top trained with variable right context. In inference time, the context length for the variable context layers can be changed to trade off the latency and the accuracy of the model. We also show that we can run this model in a Y-model architecture with the top layers running in parallel in low latency and high latency modes. This allows us to have streaming speech recognition results with limited latency and delayed speech recognition results with large improvements in accuracy (20% relative improvement for voice-search task). We show that with limited right context (1-2 seconds of audio) and small additional latency (50-100 milliseconds) at the end of decoding, we can achieve similar accuracy with models using unlimited audio right context. We also present optimizations for audio and label encoders to speed up the inference in streaming and non-streaming speech decoding.