TTS-Guided Training for Accent Conversion Without Parallel Data

TL;DR

This paper introduces a non-parallel accent conversion method using a pretrained TTS system to transform speech accents while maintaining content and speaker identity, validated through experiments on English accents.

Contribution

A novel non-parallel accent conversion framework leveraging a pretrained TTS model and speech encoder to effectively change accents without parallel data.

Findings

High-quality accent conversion achieved without parallel data.

Objective and subjective tests confirm the effectiveness of the approach.

Successful conversion of Chinese and Indian accents to American, British, and Canadian accents.

Abstract

Accent Conversion (AC) seeks to change the accent of speech from one (source) to another (target) while preserving the speech content and speaker identity. However, many AC approaches rely on source-target parallel speech data. We propose a novel accent conversion framework without the need of parallel data. Specifically, a text-to-speech (TTS) system is first pretrained with target-accented speech data. This TTS model and its hidden representations are expected to be associated only with the target accent. Then, a speech encoder is trained to convert the accent of the speech under the supervision of the pretrained TTS model. In doing so, the source-accented speech and its corresponding transcription are forwarded to the speech encoder and the pretrained TTS, respectively. The output of the speech encoder is optimized to be the same as the text embedding in the TTS system. At run-time, the speech encoder is combined with the pretrained TTS decoder to convert the source-accented speech toward the target. In the experiments, we converted English with two source accents (Chinese and Indian) to the target accent (American/British/Canadian). Both objective metrics and subjective listening tests successfully validate that, without any parallel data, the proposed approach generates speech samples that are close to the target accent with high speech quality.