Chunked Attention-based Encoder-Decoder Model for Streaming Speech Recognition

TL;DR

This paper introduces a chunked attention-based encoder-decoder model for streaming speech recognition, which operates on fixed-size chunks with an end-of-chunk symbol, achieving competitive performance and good long-form speech generalization.

Contribution

The paper proposes a novel chunked attention-based encoder-decoder architecture with an end-of-chunk symbol, bridging the gap between standard models and transducers for streaming speech recognition.

Findings

Maintains competitive performance with non-streamable models

Generalizes effectively to long-form speech

Operates efficiently on fixed-size chunks

Abstract

We study a streamable attention-based encoder-decoder model in which either the decoder, or both the encoder and decoder, operate on pre-defined, fixed-size windows called chunks. A special end-of-chunk (EOC) symbol advances from one chunk to the next chunk, effectively replacing the conventional end-of-sequence symbol. This modification, while minor, situates our model as equivalent to a transducer model that operates on chunks instead of frames, where EOC corresponds to the blank symbol. We further explore the remaining differences between a standard transducer and our model. Additionally, we examine relevant aspects such as long-form speech generalization, beam size, and length normalization. Through experiments on Librispeech and TED-LIUM-v2, and by concatenating consecutive sequences for long-form trials, we find that our streamable model maintains competitive performance compared to the non-streamable variant and generalizes very well to long-form speech.