ChunkFormer: Masked Chunking Conformer For Long-Form Speech Transcription

TL;DR

ChunkFormer is an efficient speech recognition model that enables processing of up to 16 hours of audio on low-memory GPUs, significantly reducing resource usage and improving accuracy for long-form transcription.

Contribution

It introduces chunk-wise processing with relative right context and masked batching, allowing long audio transcription on low-memory GPUs and outperforming existing models in length and accuracy.

Findings

Handles up to 16 hours of audio on an 80GB GPU

Reduces execution time and memory usage by over 3x

Achieves up to 7.7% absolute reduction in word error rate

Abstract

Deploying ASR models at an industrial scale poses significant challenges in hardware resource management, especially for long-form transcription tasks where audio may last for hours. Large Conformer models, despite their capabilities, are limited to processing only 15 minutes of audio on an 80GB GPU. Furthermore, variable input lengths worsen inefficiencies, as standard batching leads to excessive padding, increasing resource consumption and execution time. To address this, we introduce ChunkFormer, an efficient ASR model that uses chunk-wise processing with relative right context, enabling long audio transcriptions on low-memory GPUs. ChunkFormer handles up to 16 hours of audio on an 80GB GPU, 1.5x longer than the current state-of-the-art FastConformer, while also boosting long-form transcription performance with up to 7.7% absolute reduction on word error rate and maintaining accuracy on shorter tasks compared to Conformer. By eliminating the need for padding in standard batching, ChunkFormer's masked batching technique reduces execution time and memory usage by more than 3x in batch processing, substantially reducing costs for a wide range of ASR systems, particularly regarding GPU resources for models serving in real-world applications.