Modular End-to-end Automatic Speech Recognition Framework for Acoustic-to-word Model

TL;DR

This paper introduces a modular end-to-end speech recognition system that separates acoustic and language modeling, allowing the use of large text datasets to improve recognition accuracy without breaking the end-to-end property.

Contribution

A novel modular E2E ASR framework combining acoustic-to-phoneme and phoneme-to-word models, enabling the integration of extra text data during training.

Findings

Achieves lower word error rate than standard A2W models

Effectively utilizes large-scale text data for language modeling

Maintains end-to-end decoding process

Abstract

End-to-end (E2E) systems have played a more and more important role in automatic speech recognition (ASR) and achieved great performance. However, E2E systems recognize output word sequences directly with the input acoustic feature, which can only be trained on limited acoustic data. The extra text data is widely used to improve the results of traditional artificial neural network-hidden Markov model (ANN-HMM) hybrid systems. The involving of extra text data to standard E2E ASR systems may break the E2E property during decoding. In this paper, a novel modular E2E ASR system is proposed. The modular E2E ASR system consists of two parts: an acoustic-to-phoneme (A2P) model and a phoneme-to-word (P2W) model. The A2P model is trained on acoustic data, while extra data including large scale text data can be used to train the P2W model. This additional data enables the modular E2E ASR system to model not only the acoustic part but also the language part. During the decoding phase, the two models will be integrated and act as a standard acoustic-to-word (A2W) model. In other words, the proposed modular E2E ASR system can be easily trained with extra text data and decoded in the same way as a standard E2E ASR system. Experimental results on the Switchboard corpus show that the modular E2E model achieves better word error rate (WER) than standard A2W models.