SC-GlowTTS: an Efficient Zero-Shot Multi-Speaker Text-To-Speech Model

TL;DR

SC-GlowTTS is a novel zero-shot multi-speaker TTS model that leverages flow-based decoding and GAN vocoders to achieve high similarity and speech quality for unseen speakers with efficient training.

Contribution

The paper introduces a speaker-conditional flow-based decoder and a GAN vocoder adjustment, enabling zero-shot speaker adaptation in TTS with improved similarity and quality.

Findings

Achieves state-of-the-art similarity for unseen speakers.

Converges with only 11 training speakers.

Significantly improves speech quality for new speakers.

Abstract

In this paper, we propose SC-GlowTTS: an efficient zero-shot multi-speaker text-to-speech model that improves similarity for speakers unseen during training. We propose a speaker-conditional architecture that explores a flow-based decoder that works in a zero-shot scenario. As text encoders, we explore a dilated residual convolutional-based encoder, gated convolutional-based encoder, and transformer-based encoder. Additionally, we have shown that adjusting a GAN-based vocoder for the spectrograms predicted by the TTS model on the training dataset can significantly improve the similarity and speech quality for new speakers. Our model converges using only 11 speakers, reaching state-of-the-art results for similarity with new speakers, as well as high speech quality.