Training Text-to-Speech Model with Purely Synthetic Data: Feasibility, Sensitivity, and Generalization Capability

TL;DR

This paper systematically evaluates the use of purely synthetic data for training text-to-speech models, demonstrating its potential to outperform real data under certain conditions by enhancing diversity and reducing noise.

Contribution

It provides a comprehensive analysis of synthetic data's feasibility, sensitivity factors, and generalization capabilities in TTS training, highlighting key factors that improve performance.

Findings

Increasing speaker and text diversity improves quality and robustness.

Cleaner, less noisy data enhances model performance.

Models trained on synthetic data can outperform real-data models under similar conditions.

Abstract

The potential of synthetic data in text-to-speech (TTS) model training has gained increasing attention, yet its rationality and effectiveness require systematic validation. In this study, we systematically investigate the feasibility of using purely synthetic data for TTS training and explore how various factors--including text richness, speaker diversity, noise levels, and speaking styles--affect model performance. Our experiments reveal that increasing speaker and text diversity significantly enhances synthesis quality and robustness. Cleaner training data with minimal noise further improves performance. Moreover, we find that standard speaking styles facilitate more effective model learning. Our experiments indicate that models trained on synthetic data have great potential to outperform those trained on real data under similar conditions, due to the absence of real-world imperfections and noise.