MHTTS: Fast multi-head text-to-speech for spontaneous speech with imperfect transcription

TL;DR

MHTTS is a fast, multi-speaker TTS system that effectively synthesizes expressive speech from spontaneous, imperfectly transcribed data by transferring high-quality text information, improving quality and speed.

Contribution

It introduces a multi-head model that transfers text info from high-quality to imperfect transcription data, enabling efficient spontaneous speech synthesis.

Findings

Synthesizes higher quality multi-speaker speech

Faster inference speed

Effective transfer from imperfect transcriptions

Abstract

Neural network based end-to-end Text-to-Speech (TTS) has greatly improved the quality of synthesized speech. While how to use massive spontaneous speech without transcription efficiently still remains an open problem. In this paper, we propose MHTTS, a fast multi-speaker TTS system that is robust to transcription errors and speaking style speech data. Specifically, we introduce a multi-head model and transfer text information from high-quality corpus with manual transcription to spontaneous speech with imperfectly recognized transcription by jointly training them. MHTTS has three advantages: 1) Our system synthesizes better quality multi-speaker voice with faster inference speed. 2) Our system is capable of transferring correct text information to data with imperfect transcription, simulated using corruption, or provided by an Automatic Speech Recogniser (ASR). 3) Our system can utilize massive real spontaneous speech with imperfect transcription and synthesize expressive voice.