DART: Disentanglement of Accent and Speaker Representation in Multispeaker Text-to-Speech

TL;DR

This paper introduces a novel method using multi-level variational autoencoders and vector quantization to disentangle speaker and accent representations in multispeaker TTS, enabling more flexible and personalized speech synthesis.

Contribution

It presents a new approach that effectively separates speaker and accent features, improving control and personalization in multispeaker TTS systems.

Findings

Enhanced ability to control accent and speaker identity independently

Improved naturalness and diversity of synthesized speech

Public release of code and speech samples

Abstract

Recent advancements in Text-to-Speech (TTS) systems have enabled the generation of natural and expressive speech from textual input. Accented TTS aims to enhance user experience by making the synthesized speech more relatable to minority group listeners, and useful across various applications and context. Speech synthesis can further be made more flexible by allowing users to choose any combination of speaker identity and accent, resulting in a wide range of personalized speech outputs. Current models struggle to disentangle speaker and accent representation, making it difficult to accurately imitate different accents while maintaining the same speaker characteristics. We propose a novel approach to disentangle speaker and accent representations using multi-level variational autoencoders (ML-VAE) and vector quantization (VQ) to improve flexibility and enhance personalization in speech synthesis. Our proposed method addresses the challenge of effectively separating speaker and accent characteristics, enabling more fine-grained control over the synthesized speech. Code and speech samples are publicly available.