PromptSpeaker: Speaker Generation Based on Text Descriptions

TL;DR

PromptSpeaker is a novel system that generates speaker voices from text descriptions using a combination of prompt encoding, a Glow model, and zero-shot voice synthesis, enabling new speaker creation without training on specific speakers.

Contribution

It introduces a new text-guided speaker generation framework combining prompt encoding, Glow, and zero-shot VITS for flexible speaker synthesis from descriptions.

Findings

Successfully generates new speakers from text prompts.

Achieves reasonable subjective quality in voice matching.

Demonstrates effectiveness with objective metrics.

Abstract

Recently, text-guided content generation has received extensive attention. In this work, we explore the possibility of text description-based speaker generation, i.e., using text prompts to control the speaker generation process. Specifically, we propose PromptSpeaker, a text-guided speaker generation system. PromptSpeaker consists of a prompt encoder, a zero-shot VITS, and a Glow model, where the prompt encoder predicts a prior distribution based on the text description and samples from this distribution to obtain a semantic representation. The Glow model subsequently converts the semantic representation into a speaker representation, and the zero-shot VITS finally synthesizes the speaker's voice based on the speaker representation. We verify that PromptSpeaker can generate speakers new from the training set by objective metrics, and the synthetic speaker voice has reasonable subjective matching quality with the speaker prompt.