Improving Pronunciation and Accent Conversion through Knowledge Distillation And Synthetic Ground-Truth from Native TTS

TL;DR

This paper introduces a novel accent conversion method that enhances pronunciation and accent naturalness by generating synthetic ground-truth native-like speech using knowledge distillation and TTS, improving comprehensibility.

Contribution

It presents a new approach combining knowledge distillation and synthetic ground-truth generation within the VITS framework for improved accent conversion and pronunciation enhancement.

Findings

Produced native-like accent conversion with preserved speaker identity.

Improved pronunciation clarity demonstrated in evaluations.

Achieved high-quality waveform synthesis for non-native speakers.

Abstract

Previous approaches on accent conversion (AC) mainly aimed at making non-native speech sound more native while maintaining the original content and speaker identity. However, non-native speakers sometimes have pronunciation issues, which can make it difficult for listeners to understand them. Hence, we developed a new AC approach that not only focuses on accent conversion but also improves pronunciation of non-native accented speaker. By providing the non-native audio and the corresponding transcript, we generate the ideal ground-truth audio with native-like pronunciation with original duration and prosody. This ground-truth data aids the model in learning a direct mapping between accented and native speech. We utilize the end-to-end VITS framework to achieve high-quality waveform reconstruction for the AC task. As a result, our system not only produces audio that closely resembles native accents and while retaining the original speaker's identity but also improve pronunciation, as demonstrated by evaluation results.