Multi-speaker Multi-style Text-to-speech Synthesis With Single-speaker Single-style Training Data Scenarios

TL;DR

This paper introduces a novel Tacotron2-based framework for multi-speaker, multi-style text-to-speech synthesis that can combine styles and timbres across speakers, enhancing expressiveness and diversity without requiring multi-style data per speaker.

Contribution

It proposes a new method with a prosody predicting module and speaker controller, enabling style transfer across speakers using single-style data per speaker.

Findings

Successfully transfers styles between speakers.

Increases speech diversity through explicit prosody features.

Bypasses the need for multi-style recordings per speaker.

Abstract

In the existing cross-speaker style transfer task, a source speaker with multi-style recordings is necessary to provide the style for a target speaker. However, it is hard for one speaker to express all expected styles. In this paper, a more general task, which is to produce expressive speech by combining any styles and timbres from a multi-speaker corpus in which each speaker has a unique style, is proposed. To realize this task, a novel method is proposed. This method is a Tacotron2-based framework but with a fine-grained text-based prosody predicting module and a speaker identity controller. Experiments demonstrate that the proposed method can successfully express a style of one speaker with the timber of another speaker bypassing the dependency on a single speaker's multi-style corpus. Moreover, the explicit prosody features used in the prosody predicting module can increase the diversity of synthetic speech by adjusting the value of prosody features.