SSCFormer: Push the Limit of Chunk-wise Conformer for Streaming ASR Using Sequentially Sampled Chunks and Chunked Causal Convolution

TL;DR

SSCFormer advances streaming ASR by introducing a novel context generation method and chunked causal convolution, enabling better global context capture, efficient training, and linear inference complexity.

Contribution

The paper proposes SSCFormer, a new chunk-wise conformer architecture with sequential sampling and chunked causal convolution for improved streaming ASR.

Findings

Achieves 5.33% CER on AISHELL-1, outperforming baseline.

Enables training with large batch sizes and linear inference complexity.

Effectively captures long-term context in streaming ASR.

Abstract

Currently, the chunk-wise schemes are often used to make Automatic Speech Recognition (ASR) models to support streaming deployment. However, existing approaches are unable to capture the global context, lack support for parallel training, or exhibit quadratic complexity for the computation of multi-head self-attention (MHSA). On the other side, the causal convolution, no future context used, has become the de facto module in streaming Conformer. In this paper, we propose SSCFormer to push the limit of chunk-wise Conformer for streaming ASR using the following two techniques: 1) A novel cross-chunks context generation method, named Sequential Sampling Chunk (SSC) scheme, to re-partition chunks from regular partitioned chunks to facilitate efficient long-term contextual interaction within local chunks. 2)The Chunked Causal Convolution (C2Conv) is designed to concurrently capture the left context and chunk-wise future context. Evaluations on AISHELL-1 show that an End-to-End (E2E) CER 5.33% can achieve, which even outperforms a strong time-restricted baseline U2. Moreover, the chunk-wise MHSA computation in our model enables it to train with a large batch size and perform inference with linear complexity.