Accelerating RNN-T Training and Inference Using CTC guidance

TL;DR

This paper introduces a CTC-guided method to speed up RNN-T training and inference by discarding frames likely to be blank, achieving over twice the speed with comparable accuracy.

Contribution

The novel approach leverages CTC guidance to reduce frames in RNN-T, significantly accelerating training and inference without sacrificing accuracy.

Findings

RNN-T inference is accelerated by 2.2 times.

Frame reduction maintains or slightly improves WER.

Method is effective on Librispeech and SpeechStew datasets.

Abstract

We propose a novel method to accelerate training and inference process of recurrent neural network transducer (RNN-T) based on the guidance from a co-trained connectionist temporal classification (CTC) model. We made a key assumption that if an encoder embedding frame is classified as a blank frame by the CTC model, it is likely that this frame will be aligned to blank for all the partial alignments or hypotheses in RNN-T and it can be discarded from the decoder input. We also show that this frame reduction operation can be applied in the middle of the encoder, which result in significant speed up for the training and inference in RNN-T. We further show that the CTC alignment, a by-product of the CTC decoder, can also be used to perform lattice reduction for RNN-T during training. Our method is evaluated on the Librispeech and SpeechStew tasks. We demonstrate that the proposed method is able to accelerate the RNN-T inference by 2.2 times with similar or slightly better word error rates (WER).