Multi-speaker Emotional Text-to-speech Synthesizer

TL;DR

This paper introduces a multi-speaker emotional TTS system trained through a curriculum learning approach, capable of synthesizing speech with 7 emotions for 10 speakers, with efficient training and high-quality output.

Contribution

The paper presents a novel curriculum learning methodology for multi-speaker emotional TTS, enabling efficient training and synthesis of diverse speaker-emotion combinations.

Findings

Efficient training achieved through curriculum learning and silence removal.

Model can synthesize speech for 10 speakers with 7 emotions.

Synthesized speech quality is suitable for practical applications.

Abstract

We present a methodology to train our multi-speaker emotional text-to-speech synthesizer that can express speech for 10 speakers' 7 different emotions. All silences from audio samples are removed prior to learning. This results in fast learning by our model. Curriculum learning is applied to train our model efficiently. Our model is first trained with a large single-speaker neutral dataset, and then trained with neutral speech from all speakers. Finally, our model is trained using datasets of emotional speech from all speakers. In each stage, training samples of each speaker-emotion pair have equal probability to appear in mini-batches. Through this procedure, our model can synthesize speech for all targeted speakers and emotions. Our synthesized audio sets are available on our web page.