Improved Mask-CTC for Non-Autoregressive End-to-End ASR

TL;DR

This paper enhances the Mask-CTC non-autoregressive ASR system by integrating Conformer architecture and auxiliary objectives, significantly improving accuracy while maintaining fast inference speed, and demonstrating potential in speech translation.

Contribution

The paper introduces architectural and training improvements to Mask-CTC, achieving higher accuracy without sacrificing inference speed, and explores its application to speech translation.

Findings

WER reduced from 15.5% to 9.1% on WSJ

Achieves competitive results to autoregressive models

Maintains inference speed with less than 0.1 RTF

Abstract

For real-world deployment of automatic speech recognition (ASR), the system is desired to be capable of fast inference while relieving the requirement of computational resources. The recently proposed end-to-end ASR system based on mask-predict with connectionist temporal classification (CTC), Mask-CTC, fulfills this demand by generating tokens in a non-autoregressive fashion. While Mask-CTC achieves remarkably fast inference speed, its recognition performance falls behind that of conventional autoregressive (AR) systems. To boost the performance of Mask-CTC, we first propose to enhance the encoder network architecture by employing a recently proposed architecture called Conformer. Next, we propose new training and decoding methods by introducing auxiliary objective to predict the length of a partial target sequence, which allows the model to delete or insert tokens during inference. Experimental results on different ASR tasks show that the proposed approaches improve Mask-CTC significantly, outperforming a standard CTC model (15.5% $\rightarrow$ 9.1% WER on WSJ). Moreover, Mask-CTC now achieves competitive results to AR models with no degradation of inference speed ($<$ 0.1 RTF using CPU). We also show a potential application of Mask-CTC to end-to-end speech translation.