ESPnet2-TTS: Extending the Edge of TTS Research

TL;DR

ESPnet2-TTS is an advanced, unified toolkit for end-to-end text-to-speech synthesis that simplifies training, offers state-of-the-art models, and provides pre-trained models for quick deployment and research.

Contribution

It introduces new features like flexible pre-processing, joint training with neural vocoders, and full-band modeling, enhancing TTS performance and usability over previous versions.

Findings

Achieves state-of-the-art TTS performance on English and Japanese datasets.

Provides pre-trained models for quick inference and demo creation.

Simplifies the training pipeline with unified recipes.

Abstract

This paper describes ESPnet2-TTS, an end-to-end text-to-speech (E2E-TTS) toolkit. ESPnet2-TTS extends our earlier version, ESPnet-TTS, by adding many new features, including: on-the-fly flexible pre-processing, joint training with neural vocoders, and state-of-the-art TTS models with extensions like full-band E2E text-to-waveform modeling, which simplify the training pipeline and further enhance TTS performance. The unified design of our recipes enables users to quickly reproduce state-of-the-art E2E-TTS results. We also provide many pre-trained models in a unified Python interface for inference, offering a quick means for users to generate baseline samples and build demos. Experimental evaluations with English and Japanese corpora demonstrate that our provided models synthesize utterances comparable to ground-truth ones, achieving state-of-the-art TTS performance. The toolkit is available online at https://github.com/espnet/espnet.