Multi-reference Tacotron by Intercross Training for Style Disentangling,Transfer and Control in Speech Synthesis

TL;DR

This paper introduces a multi-reference Tacotron model with intercross training that enables independent control and transfer of specific speech styles, enhancing speech synthesis diversity and expressiveness.

Contribution

It presents a novel multi-reference structure and intercross training method that disentangles and independently controls individual speech style features in synthesis.

Findings

Effective independent control of speech styles demonstrated

Successful style transfer in synthesized speech

Enhanced diversity and expressiveness in speech synthesis

Abstract

Speech style control and transfer techniques aim to enrich the diversity and expressiveness of synthesized speech. Existing approaches model all speech styles into one representation, lacking the ability to control a specific speech feature independently. To address this issue, we introduce a novel multi-reference structure to Tacotron and propose intercross training approach, which together ensure that each sub-encoder of the multi-reference encoder independently disentangles and controls a specific style. Experimental results show that our model is able to control and transfer desired speech styles individually.