The NTU-AISG Text-to-speech System for Blizzard Challenge 2020

TL;DR

This paper presents NTU-AISG's low-resource Mandarin and Shanghai dialect TTS systems for Blizzard Challenge 2020, utilizing external data and average-speaker modeling to address data scarcity, with promising naturalness but some intelligibility issues.

Contribution

The study introduces a novel average-speaker modeling approach for low-resource dialect TTS, leveraging external Mandarin data for training and adaptation.

Findings

Good naturalness and speaker similarity in synthesized speech.

Intelligibility of Shanghai dialect TTS is significantly challenged.

Effective adaptation of Mandarin-trained models to Shanghai dialect with limited data.

Abstract

We report our NTU-AISG Text-to-speech (TTS) entry systems for the Blizzard Challenge 2020 in this paper. There are two TTS tasks in this year's challenge, one is a Mandarin TTS task, the other is a Shanghai dialect TTS task. We have participated both. One of the main challenges is to build TTS systems with low-resource constraints, particularly for the case of Shanghai dialect, of which about three hours data are available to participants. To overcome the constraint, we adopt an average-speaker modeling method. That is, we first employ external Mandarin data to train both End-to-end acoustic model and WaveNet vocoder, then we use Shanghai dialect to tune the acoustic model and WaveNet vocoder respectively. Apart from this, we have no Shanghai dialect lexicon despite syllable transcripts are provided for the training data. Since we are not sure if similar syllable transcripts are provided for the evaluation data during the training stage, we use Mandarin lexicon for Shanghai dialect instead. With the letter, as decomposed from the corresponding Mandarin syllable, as input, though the naturalness and original speaker similarity of the synthesized speech are good, subjective evaluation results indicate the intelligibility of the synthesized speech is deeply undermined for the Shanghai dialect TTS system.