Spoken Language Corpora Augmentation with Domain-Specific Voice-Cloned Speech

TL;DR

This study investigates how augmenting spoken language datasets with synthetic, domain-specific voice-cloned speech affects speech recognition performance, highlighting the benefits of high-variability synthetic data over low-variability data.

Contribution

It demonstrates that high-variability synthetic speech, generated via voice cloning, significantly improves speech recognition accuracy compared to low-variability synthetic data.

Findings

High-variability synthetic speech enhances ASR performance.

Low-variability synthetic data quickly saturates in effectiveness.

Voice cloning with multiple voices outperforms conventional TTS in augmentation.

Abstract

In this paper we study the impact of augmenting spoken language corpora with domain-specific synthetic samples for the purpose of training a speech recognition system. Using both a conventional neural TTS system and a zero-shot one with voice cloning ability we generate speech corpora that vary in the number of voices. We compare speech recognition models trained with addition of different amounts of synthetic data generated using these two methods with a baseline model trained solely on voice recordings. We show that while the quality of voice-cloned dataset is lower, its increased multivoiceity makes it much more effective than the one with only a few voices synthesized with the use of a conventional neural TTS system. Furthermore, our experiments indicate that using low variability synthetic speech quickly leads to saturation in the quality of the ASR whereas high variability speech provides improvement even when increasing total amount of data used for training by 30%.