On lattice-free boosted MMI training of HMM and CTC-based full-context ASR models

TL;DR

This paper introduces a unified framework for hybrid ASR training using LF-MMI across various modeling units and topologies, proposing novel training schemes that improve performance and efficiency.

Contribution

It generalizes LF-MMI training to full-context models and multiple units, and proposes three new training schemes with demonstrated advantages.

Findings

LF-MMI is effective for both limited and full-context models.

Proposed schemes improve training performance and decoding efficiency.

Bi-char HMM-MMI models outperform traditional GMM-HMMs as alignment models.

Abstract

Hybrid automatic speech recognition (ASR) models are typically sequentially trained with CTC or LF-MMI criteria. However, they have vastly different legacies and are usually implemented in different frameworks. In this paper, by decoupling the concepts of modeling units and label topologies and building proper numerator/denominator graphs accordingly, we establish a generalized framework for hybrid acoustic modeling (AM). In this framework, we show that LF-MMI is a powerful training criterion applicable to both limited-context and full-context models, for wordpiece/mono-char/bi-char/chenone units, with both HMM/CTC topologies. From this framework, we propose three novel training schemes: chenone(ch)/wordpiece(wp)-CTC-bMMI, and wordpiece(wp)-HMM-bMMI with different advantages in training performance, decoding efficiency and decoding time-stamp accuracy. The advantages of different training schemes are evaluated comprehensively on Librispeech, and wp-CTC-bMMI and ch-CTC-bMMI are evaluated on two real world ASR tasks to show their effectiveness. Besides, we also show bi-char(bc) HMM-MMI models can serve as better alignment models than traditional non-neural GMM-HMMs.