Unsupervised Quantized Prosody Representation for Controllable Speech Synthesis

TL;DR

This paper introduces an unsupervised, vector quantization-based prosody disentanglement method for controllable speech synthesis, enabling automatic style decomposition and manipulation in TTS systems.

Contribution

It presents a novel VQ-based prosody encoder for unsupervised style disentanglement and demonstrates effective control over speaking styles in TTS.

Findings

Model outperforms existing methods in objective evaluations.

Subjective tests confirm improved controllability of speaking styles.

Higher VQ dimensions encode prosody information.

Abstract

In this paper, we propose a novel prosody disentangle method for prosodic Text-to-Speech (TTS) model, which introduces the vector quantization (VQ) method to the auxiliary prosody encoder to obtain the decomposed prosody representations in an unsupervised manner. Rely on its advantages, the speaking styles, such as pitch, speaking velocity, local pitch variance, etc., are decomposed automatically into the latent quantize vectors. We also investigate the internal mechanism of VQ disentangle process by means of a latent variables counter and find that higher value dimensions usually represent prosody information. Experiments show that our model can control the speaking styles of synthesis results by directly manipulating the latent variables. The objective and subjective evaluations illustrated that our model outperforms the popular models.