Improving the quality of neural TTS using long-form content and multi-speaker multi-style modeling

TL;DR

This paper enhances neural TTS quality by leveraging long-form recordings and multi-speaker multi-style modeling, enabling style transfer and improved naturalness with less data.

Contribution

It introduces a multi-speaker multi-style model trained on long-form data, outperforming traditional pre-training methods for style transfer and quality enhancement.

Findings

Multi-speaker modeling improves TTS quality.

MSMS approach outperforms pre-training and fine-tuning.

Long-form style is highly rated across domains.

Abstract

Neural text-to-speech (TTS) can provide quality close to natural speech if an adequate amount of high-quality speech material is available for training. However, acquiring speech data for TTS training is costly and time-consuming, especially if the goal is to generate different speaking styles. In this work, we show that we can transfer speaking style across speakers and improve the quality of synthetic speech by training a multi-speaker multi-style (MSMS) model with long-form recordings, in addition to regular TTS recordings. In particular, we show that 1) multi-speaker modeling improves the overall TTS quality, 2) the proposed MSMS approach outperforms pre-training and fine-tuning approach when utilizing additional multi-speaker data, and 3) long-form speaking style is highly rated regardless of the target text domain.