Improving Data Augmentation-based Cross-Speaker Style Transfer for TTS with Singing Voice, Style Filtering, and F0 Matching

TL;DR

This paper enhances cross-speaker style transfer in TTS by integrating singing voice conversion, style filtering, and F0 matching, achieving high-quality results with minimal target data.

Contribution

It introduces a novel combination of singing voice conversion, style filtering, and F0 matching to improve style transfer efficiency and quality in TTS.

Findings

Comparable to state-of-the-art with only a few minutes of target data

Improved naturalness and style intensity in perceptual tests

Increased speaker similarity with F0 matching

Abstract

The goal of cross-speaker style transfer in TTS is to transfer a speech style from a source speaker with expressive data to a target speaker with only neutral data. In this context, we propose using a pre-trained singing voice conversion (SVC) model to convert the expressive data into the target speaker's voice. In the conversion process, we apply a fundamental frequency (F0) matching technique to mitigate tonal variances between speakers with significant timbral differences. A style classifier filter is proposed to select the most expressive output audios for the TTS training. Our approach is comparable to state-of-the-art with only a few minutes of neutral data from the target speaker, while other methods require hours. A perceptual assessment showed improvements brought by the SVC and the style filter in naturalness and style intensity for the styles that display more vocal effort. Also, increased speaker similarity is obtained with the proposed F0 matching algorithm.