DreamVoice: Text-Guided Voice Conversion

TL;DR

DreamVoice introduces a new dataset and two text-guided voice conversion methods that enable intuitive, high-quality voice transformation aligned with textual prompts, enhancing personalization and ease of use.

Contribution

The paper presents DreamVoiceDB dataset and two novel text-guided VC methods, DreamVC and DreamVG, advancing voice conversion technology with text-based control.

Findings

High-quality voice conversion aligned with text prompts

Effective voice timbre generation for 900 speakers

Versatile plugin compatible with existing VC models

Abstract

Generative voice technologies are rapidly evolving, offering opportunities for more personalized and inclusive experiences. Traditional one-shot voice conversion (VC) requires a target recording during inference, limiting ease of usage in generating desired voice timbres. Text-guided generation offers an intuitive solution to convert voices to desired "DreamVoices" according to the users' needs. Our paper presents two major contributions to VC technology: (1) DreamVoiceDB, a robust dataset of voice timbre annotations for 900 speakers from VCTK and LibriTTS. (2) Two text-guided VC methods: DreamVC, an end-to-end diffusion-based text-guided VC model; and DreamVG, a versatile text-to-voice generation plugin that can be combined with any one-shot VC models. The experimental results demonstrate that our proposed methods trained on the DreamVoiceDB dataset generate voice timbres accurately aligned with the text prompt and achieve high-quality VC.