LSTM Acoustic Models Learn to Align and Pronounce with Graphemes

TL;DR

This paper introduces a data-driven, grapheme-based LSTM acoustic model for speech recognition that aligns and pronounces words without needing handcrafted phoneme lexicons, showing competitive accuracy and useful alignments.

Contribution

It presents a novel training methodology for grapheme-based speech recognition models that do not require linguistic lexicons, enabling practical and scalable ASR systems.

Findings

Grapheme models achieve comparable WER to phoneme models on large datasets.

Grapheme models produce high-quality audio-to-grapheme alignments.

The approach works effectively across linguistically diverse Indian languages.

Abstract

Automated speech recognition coverage of the world's languages continues to expand. However, standard phoneme based systems require handcrafted lexicons that are difficult and expensive to obtain. To address this problem, we propose a training methodology for a grapheme-based speech recognizer that can be trained in a purely data-driven fashion. Built with LSTM networks and trained with the cross-entropy loss, the grapheme-output acoustic models we study are also extremely practical for real-world applications as they can be decoded with conventional ASR stack components such as language models and FST decoders, and produce good quality audio-to-grapheme alignments that are useful in many speech applications. We show that the grapheme models are competitive in WER with their phoneme-output counterparts when trained on large datasets, with the advantage that grapheme models do not require explicit linguistic knowledge as an input. We further compare the alignments generated by the phoneme and grapheme models to demonstrate the quality of the pronunciations learnt by them using four Indian languages that vary linguistically in spoken and written forms.