MuLanTTS: The Microsoft Speech Synthesis System for Blizzard Challenge 2023

TL;DR

MuLanTTS is a neural TTS system that leverages contextual and emotion encoders, achieving high-quality speech synthesis for French audiobook data in the Blizzard Challenge 2023.

Contribution

It introduces MuLanTTS, an end-to-end neural TTS system with enhanced prosody and expressiveness, adapted for long-form and dialogue speech synthesis.

Findings

Achieved mean quality scores of 4.3 and 4.5, comparable to natural speech.

Effectively used denoise algorithms and long audio processing.

Demonstrated strong performance in both hub and spoke tasks.

Abstract

In this paper, we present MuLanTTS, the Microsoft end-to-end neural text-to-speech (TTS) system designed for the Blizzard Challenge 2023. About 50 hours of audiobook corpus for French TTS as hub task and another 2 hours of speaker adaptation as spoke task are released to build synthesized voices for different test purposes including sentences, paragraphs, homographs, lists, etc. Building upon DelightfulTTS, we adopt contextual and emotion encoders to adapt the audiobook data to enrich beyond sentences for long-form prosody and dialogue expressiveness. Regarding the recording quality, we also apply denoise algorithms and long audio processing for both corpora. For the hub task, only the 50-hour single speaker data is used for building the TTS system, while for the spoke task, a multi-speaker source model is used for target speaker fine tuning. MuLanTTS achieves mean scores of quality assessment 4.3 and 4.5 in the respective tasks, statistically comparable with natural speech while keeping good similarity according to similarity assessment. The excellent and similarity in this year's new and dense statistical evaluation show the effectiveness of our proposed system in both tasks.