HAINAN: Fast and Accurate Transducer for Hybrid-Autoregressive ASR

TL;DR

HAINAN introduces a flexible speech recognition model that combines autoregressive and non-autoregressive inference, achieving high accuracy and efficiency across multiple languages and datasets.

Contribution

The paper proposes HAINAN, a novel hybrid-inference transducer architecture that supports both inference modes and introduces a semi-autoregressive paradigm for improved accuracy.

Findings

Achieves efficiency parity with CTC and TDT in respective modes.

Outperforms TDT and RNN-T in autoregressive mode.

Significantly outperforms CTC in non-autoregressive mode.

Abstract

We present Hybrid-Autoregressive INference TrANsducers (HAINAN), a novel architecture for speech recognition that extends the Token-and-Duration Transducer (TDT) model. Trained with randomly masked predictor network outputs, HAINAN supports both autoregressive inference with all network components and non-autoregressive inference without the predictor. Additionally, we propose a novel semi-autoregressive inference paradigm that first generates an initial hypothesis using non-autoregressive inference, followed by refinement steps where each token prediction is regenerated using parallelized autoregression on the initial hypothesis. Experiments on multiple datasets across different languages demonstrate that HAINAN achieves efficiency parity with CTC in non-autoregressive mode and with TDT in autoregressive mode. In terms of accuracy, autoregressive HAINAN outperforms TDT and RNN-T, while non-autoregressive HAINAN significantly outperforms CTC. Semi-autoregressive inference further enhances the model's accuracy with minimal computational overhead, and even outperforms TDT results in some cases. These results highlight HAINAN's flexibility in balancing accuracy and speed, positioning it as a strong candidate for real-world speech recognition applications.