Chunkwise Aligners for Streaming Speech Recognition

TL;DR

The paper introduces the Chunkwise Aligner, a new streaming ASR architecture that improves training and decoding efficiency while maintaining accuracy comparable to standard models.

Contribution

It presents a novel chunkwise alignment method that enables efficient streaming speech recognition without sacrificing accuracy.

Findings

Matches Transducer accuracy in offline and streaming modes.

Offers better training and decoding efficiency.

Effectively manages chunk transitions with learned probabilities.

Abstract

We propose the Chunkwise Aligner, a novel architecture for streaming automatic speech recognition (ASR). While the Transducer is the standard model for streaming ASR, its training is costly due to the need to compute all possible audio-label alignments. The recently introduced Aligner reduces this cost by discarding explicit alignments, but this modification makes it unsuitable for streaming. Our approach overcomes this limitation by dividing the audio into chunks and aligning each label to the leftmost frames of its chunk, whereas transitions between chunks are managed by a learned end-of-chunk probability. Experiments show that the Chunkwise Aligner not only matches the Transducer's accuracy in both offline and streaming scenarios, but also offers superior training and decoding efficiencies.