Unifying Streaming and Non-streaming Zipformer-based ASR
Bidisha Sharma, Karthik Pandia Durai, Shankar Venkatesan, Jeena J Prakash, Shashi Kumar, Malolan Chetlur, Andreas Stolcke

TL;DR
This paper introduces a unified end-to-end ASR model that effectively combines streaming and non-streaming capabilities using dynamic right-context and chunked attention, achieving high accuracy with controlled latency.
Contribution
It proposes a novel framework leveraging right-context in zipformer models to unify streaming and non-streaming ASR, reducing costs and improving flexibility.
Findings
Achieves 7.9% relative word error rate reduction.
Streaming performance approaches non-streaming accuracy with more right-context.
Flexible latency-accuracy tradeoff controlled by right-context size.
Abstract
There has been increasing interest in unifying streaming and non-streaming automatic speech recognition (ASR) models to reduce development, training, and deployment costs. We present a unified framework that trains a single end-to-end ASR model for both streaming and non-streaming applications, leveraging future context information. We propose to use dynamic right-context through the chunked attention masking in the training of zipformer-based ASR models. We demonstrate that using right-context is more effective in zipformer models compared to other conformer models due to its multi-scale nature. We analyze the effect of varying the number of right-context frames on accuracy and latency of the streaming ASR models. We use Librispeech and large in-house conversational datasets to train different versions of streaming and non-streaming models and evaluate them in a production grade…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsEnergy Efficient Wireless Sensor Networks · Blind Source Separation Techniques · Machine Learning and ELM
