Zero-Shot vs. Few-Shot Multi-Speaker TTS Using Pre-trained Czech SpeechT5 Model

TL;DR

This paper demonstrates that a pre-trained SpeechT5 model can generate high-quality, speaker-specific synthetic speech in Czech using only one minute of target speaker data, effective in zero- and few-shot scenarios.

Contribution

The study shows the successful fine-tuning of SpeechT5 for multi-speaker TTS in Czech, achieving high-quality synthesis with minimal data in zero- and few-shot settings.

Findings

Synthetic voices closely resemble real speakers

High-quality speech generated with only one minute of data

Effective zero- and few-shot TTS performance

Abstract

In this paper, we experimented with the SpeechT5 model pre-trained on large-scale datasets. We pre-trained the foundation model from scratch and fine-tuned it on a large-scale robust multi-speaker text-to-speech (TTS) task. We tested the model capabilities in a zero- and few-shot scenario. Based on two listening tests, we evaluated the synthetic audio quality and the similarity of how synthetic voices resemble real voices. Our results showed that the SpeechT5 model can generate a synthetic voice for any speaker using only one minute of the target speaker's data. We successfully demonstrated the high quality and similarity of our synthetic voices on publicly known Czech politicians and celebrities.