The DeepZen Speech Synthesis System for Blizzard Challenge 2023

TL;DR

The DeepZen system for Blizzard Challenge 2023 employs an auto-regressive TTS model with advanced prosodic control and style modeling, achieving high-quality French speech synthesis from large and small datasets.

Contribution

Introduces a unified auto-regressive TTS architecture with style and pronunciation modeling using BERT, improving naturalness and control in speech synthesis.

Findings

Achieved second place in both tasks with median scores of 0.75 and 0.68.

Demonstrated effective style transfer and pronunciation prediction.

Performed well with both large and small datasets.

Abstract

This paper describes the DeepZen text to speech (TTS) system for Blizzard Challenge 2023. The goal of this challenge is to synthesise natural and high-quality speech in French, from a large monospeaker dataset (hub task) and from a smaller dataset by speaker adaptation (spoke task). We participated to both tasks with the same model architecture. Our approach has been to use an auto-regressive model, which retains an advantage for generating natural sounding speech but to improve prosodic control in several ways. Similarly to non-attentive Tacotron, the model uses a duration predictor and gaussian upsampling at inference, but with a simpler unsupervised training. We also model the speaking style at both sentence and word levels by extracting global and local style tokens from the reference speech. At inference, the global and local style tokens are predicted from a BERT model run on text. This BERT model is also used to predict specific pronunciation features like schwa elision and optional liaisons. Finally, a modified version of HifiGAN trained on a large public dataset and fine-tuned on the target voices is used to generate speech waveform. Our team is identified as O in the the Blizzard evaluation and MUSHRA test results show that our system performs second ex aequo in both hub task (median score of 0.75) and spoke task (median score of 0.68), over 18 and 14 participants, respectively.