Breaking Through the Spike: Spike Window Decoding for Accelerated and Precise Automatic Speech Recognition

TL;DR

This paper introduces Spike Window Decoding, a novel method that accelerates end-to-end speech recognition by leveraging the spike property of CTC outputs, achieving state-of-the-art accuracy with faster inference.

Contribution

It proposes the Spike Window Decoding algorithm, which significantly speeds up WFST-based speech recognition by focusing on spiking frames, maintaining high accuracy.

Findings

Achieves state-of-the-art recognition accuracy.

Significantly accelerates decoding speed.

Effective across multiple datasets.

Abstract

Recently, end-to-end automatic speech recognition has become the mainstream approach in both industry and academia. To optimize system performance in specific scenarios, the Weighted Finite-State Transducer (WFST) is extensively used to integrate acoustic and language models, leveraging its capacity to implicitly fuse language models within static graphs, thereby ensuring robust recognition while also facilitating rapid error correction. However, WFST necessitates a frame-by-frame search of CTC posterior probabilities through autoregression, which significantly hampers inference speed. In this work, we thoroughly investigate the spike property of CTC outputs and further propose the conjecture that adjacent frames to non-blank spikes carry semantic information beneficial to the model. Building on this, we propose the Spike Window Decoding algorithm, which greatly improves the inference speed by making the number of frames decoded in WFST linearly related to the number of spiking frames in the CTC output, while guaranteeing the recognition performance. Our method achieves SOTA recognition accuracy with significantly accelerates decoding speed, proven across both AISHELL-1 and large-scale In-House datasets, establishing a pioneering approach for integrating CTC output with WFST.