CUSIDE: Chunking, Simulating Future Context and Decoding for Streaming ASR

TL;DR

CUSIDE introduces a novel framework for streaming ASR that uses a simulation module to generate future context, reducing latency while maintaining accuracy, and achieves state-of-the-art results on AISHELL-1.

Contribution

The paper proposes a new framework with a simulation module for future context in streaming ASR, reducing latency without sacrificing accuracy.

Findings

Simulated future context reduces latency significantly.

State-of-the-art results on AISHELL-1 dataset.

Joint training improves recognition performance.

Abstract

History and future contextual information are known to be important for accurate acoustic modeling. However, acquiring future context brings latency for streaming ASR. In this paper, we propose a new framework - Chunking, Simulating Future Context and Decoding (CUSIDE) for streaming speech recognition. A new simulation module is introduced to recursively simulate the future contextual frames, without waiting for future context. The simulation module is jointly trained with the ASR model using a self-supervised loss; the ASR model is optimized with the usual ASR loss, e.g., CTC-CRF as used in our experiments. Experiments show that, compared to using real future frames as right context, using simulated future context can drastically reduce latency while maintaining recognition accuracy. With CUSIDE, we obtain new state-of-the-art streaming ASR results on the AISHELL-1 dataset.