Mix and Match: An Empirical Study on Training Corpus Composition for Polyglot Text-To-Speech (TTS)

TL;DR

This study investigates how different factors of training data, like language family, gender, and speaker count, influence the quality of multilingual Text-To-Speech systems, providing insights for better data collection.

Contribution

It offers an extensive ablation analysis on training corpus composition factors affecting Polyglot TTS quality, highlighting practical data procurement guidelines.

Findings

Female speaker data generally yields better synthesis quality.

More speakers from the target language do not always improve performance.

Language closeness does not guarantee better transfer in multilingual TTS.

Abstract

Training multilingual Neural Text-To-Speech (NTTS) models using only monolingual corpora has emerged as a popular way for building voice cloning based Polyglot NTTS systems. In order to train these models, it is essential to understand how the composition of the training corpora affects the quality of multilingual speech synthesis. In this context, it is common to hear questions such as "Would including more Spanish data help my Italian synthesis, given the closeness of both languages?". Unfortunately, we found existing literature on the topic lacking in completeness in this regard. In the present work, we conduct an extensive ablation study aimed at understanding how various factors of the training corpora, such as language family affiliation, gender composition, and the number of speakers, contribute to the quality of Polyglot synthesis. Our findings include the observation that female speaker data are preferred in most scenarios, and that it is not always beneficial to have more speakers from the target language variant in the training corpus. The findings herein are informative for the process of data procurement and corpora building.