Generating Synthetic Audio Data for Attention-Based Speech Recognition Systems

TL;DR

This paper demonstrates that synthetic audio generated by TTS systems trained on ASR data can significantly improve attention-based speech recognition performance, especially in low-resource settings, without altering existing models.

Contribution

It introduces a method to enhance end-to-end ASR systems using TTS-generated synthetic audio trained solely on ASR data, avoiding changes to model architecture.

Findings

Up to 33% relative WER reduction on LibriSpeech-100h

Over 5% relative WER improvement on LibriSpeech-960h

Performance gains are independent of language model integration and data augmentation

Abstract

Recent advances in text-to-speech (TTS) led to the development of flexible multi-speaker end-to-end TTS systems. We extend state-of-the-art attention-based automatic speech recognition (ASR) systems with synthetic audio generated by a TTS system trained only on the ASR corpora itself. ASR and TTS systems are built separately to show that text-only data can be used to enhance existing end-to-end ASR systems without the necessity of parameter or architecture changes. We compare our method with language model integration of the same text data and with simple data augmentation methods like SpecAugment and show that performance improvements are mostly independent. We achieve improvements of up to 33% relative in word-error-rate (WER) over a strong baseline with data-augmentation in a low-resource environment (LibriSpeech-100h), closing the gap to a comparable oracle experiment by more than 50\%. We also show improvements of up to 5% relative WER over our most recent ASR baseline on LibriSpeech-960h.