Low-resource expressive text-to-speech using data augmentation

TL;DR

This paper introduces a three-step data augmentation approach for low-resource expressive TTS, enabling high-quality speech synthesis with only 15 minutes of target recordings by leveraging voice conversion and fine-tuning.

Contribution

The paper presents a novel methodology combining voice conversion, data augmentation, and fine-tuning to build expressive TTS models with minimal target data, improving quality and robustness.

Findings

Significant quality improvements over non-augmented models

Effective across multiple styles and speakers

Robust for both single and multi-speaker TTS

Abstract

While recent neural text-to-speech (TTS) systems perform remarkably well, they typically require a substantial amount of recordings from the target speaker reading in the desired speaking style. In this work, we present a novel 3-step methodology to circumvent the costly operation of recording large amounts of target data in order to build expressive style voices with as little as 15 minutes of such recordings. First, we augment data via voice conversion by leveraging recordings in the desired speaking style from other speakers. Next, we use that synthetic data on top of the available recordings to train a TTS model. Finally, we fine-tune that model to further increase quality. Our evaluations show that the proposed changes bring significant improvements over non-augmented models across many perceived aspects of synthesised speech. We demonstrate the proposed approach on 2 styles (newscaster and conversational), on various speakers, and on both single and multi-speaker models, illustrating the robustness of our approach.