ClariNet: Parallel Wave Generation in End-to-End Text-to-Speech

TL;DR

This paper introduces ClariNet, a fully convolutional, end-to-end neural architecture for fast, parallel speech waveform generation from text, improving efficiency and quality over previous methods.

Contribution

It presents a novel distillation approach for parallel WaveNet using Gaussian inverse autoregressive flow and introduces the first end-to-end text-to-wave neural speech synthesis model.

Findings

Significantly faster speech synthesis than previous pipeline methods.

Efficient training through closed-form KL divergence calculation.

Successful distillation of a parallel waveform synthesizer conditioned on hidden representations.

Abstract

In this work, we propose a new solution for parallel wave generation by WaveNet. In contrast to parallel WaveNet (van den Oord et al., 2018), we distill a Gaussian inverse autoregressive flow from the autoregressive WaveNet by minimizing a regularized KL divergence between their highly-peaked output distributions. Our method computes the KL divergence in closed-form, which simplifies the training algorithm and provides very efficient distillation. In addition, we introduce the first text-to-wave neural architecture for speech synthesis, which is fully convolutional and enables fast end-to-end training from scratch. It significantly outperforms the previous pipeline that connects a text-to-spectrogram model to a separately trained WaveNet (Ping et al., 2018). We also successfully distill a parallel waveform synthesizer conditioned on the hidden representation in this end-to-end model.