Improve Cross-lingual Voice Cloning Using Low-quality Code-switched Data

TL;DR

This paper introduces a method for cross-lingual voice cloning that leverages low-quality, code-switched data from non-target speakers, achieving high-quality, natural-sounding speech in target voices without requiring high-quality multi-lingual data.

Contribution

The paper proposes a novel approach using low-quality code-switched data from non-target speakers to improve cross-lingual voice cloning, reducing data collection costs.

Findings

Achieves high naturalness and speaker consistency in generated speech.

Comparable performance to state-of-the-art methods in cross-lingual voice cloning.

Utilizes low-quality, code-switched data effectively for multi-lingual TTS.

Abstract

Recently, sequence-to-sequence (seq-to-seq) models have been successfully applied in text-to-speech (TTS) to synthesize speech for single-language text. To synthesize speech for multiple languages usually requires multi-lingual speech from the target speaker. However, it is both laborious and expensive to collect high-quality multi-lingual TTS data for the target speakers. In this paper, we proposed to use low-quality code-switched found data from the non-target speakers to achieve cross-lingual voice cloning for the target speakers. Experiments show that our proposed method can generate high-quality code-switched speech in the target voices in terms of both naturalness and speaker consistency. More importantly, we find that our method can achieve a comparable result to the state-of-the-art (SOTA) performance in cross-lingual voice cloning.