Lightweight Transducer Based on Frame-Level Criterion

TL;DR

This paper introduces a lightweight transducer model based on frame-level criteria that reduces memory and computation by leveraging CTC alignments, while maintaining or improving performance compared to traditional transducers.

Contribution

The proposed model uses frame-level alignment and decouples blank/non-blank probabilities to create a more efficient transducer with comparable or better accuracy.

Findings

Achieves similar results to traditional transducers on AISHELL-1

Reduces memory and computation requirements significantly

Uses richer information for blank probability prediction

Abstract

The transducer model trained based on sequence-level criterion requires a lot of memory due to the generation of the large probability matrix. We proposed a lightweight transducer model based on frame-level criterion, which uses the results of the CTC forced alignment algorithm to determine the label for each frame. Then the encoder output can be combined with the decoder output at the corresponding time, rather than adding each element output by the encoder to each element output by the decoder as in the transducer. This significantly reduces memory and computation requirements. To address the problem of imbalanced classification caused by excessive blanks in the label, we decouple the blank and non-blank probabilities and truncate the gradient of the blank classifier to the main network. Experiments on the AISHELL-1 demonstrate that this enables the lightweight transducer to achieve similar results to transducer. Additionally, we use richer information to predict the probability of blank, achieving superior results to transducer.