Consistent Training and Decoding For End-to-end Speech Recognition Using Lattice-free MMI

TL;DR

This paper introduces a novel method to incorporate the LF-MMI discriminative training criterion into end-to-end speech recognition models, improving their accuracy across multiple datasets and frameworks.

Contribution

It presents the first integration of LF-MMI into E2E ASR frameworks during training and decoding, leading to consistent performance gains.

Findings

Achieved a CER of 4.1 extbackslash / 4.4 extbackslash on Aishell-1

Significant error reduction on Aishell-2 and Librispeech

Demonstrated effectiveness across AEDs and Neural Transducers

Abstract

Recently, End-to-End (E2E) frameworks have achieved remarkable results on various Automatic Speech Recognition (ASR) tasks. However, Lattice-Free Maximum Mutual Information (LF-MMI), as one of the discriminative training criteria that show superior performance in hybrid ASR systems, is rarely adopted in E2E ASR frameworks. In this work, we propose a novel approach to integrate LF-MMI criterion into E2E ASR frameworks in both training and decoding stages. The proposed approach shows its effectiveness on two of the most widely used E2E frameworks including Attention-Based Encoder-Decoders (AEDs) and Neural Transducers (NTs). Experiments suggest that the introduction of the LF-MMI criterion consistently leads to significant performance improvements on various datasets and different E2E ASR frameworks. The best of our models achieves competitive CER of 4.1\% / 4.4\% on Aishell-1 dev/test set; we also achieve significant error reduction on Aishell-2 and Librispeech datasets over strong baselines.