OverFlow: Putting flows on top of neural transducers for better TTS

TL;DR

This paper introduces OverFlow, a neural TTS model combining neural HMMs with normalising flows, resulting in a fully probabilistic, efficient, and high-quality speech synthesis system trained via maximum likelihood.

Contribution

It proposes integrating normalising flows with neural HMMs in TTS, enabling better modeling of speech acoustics and durations with fewer training updates.

Findings

Requires fewer updates for accurate pronunciation

Achieves near-natural speech quality

Fully probabilistic model trained with maximum likelihood

Abstract

Neural HMMs are a type of neural transducer recently proposed for sequence-to-sequence modelling in text-to-speech. They combine the best features of classic statistical speech synthesis and modern neural TTS, requiring less data and fewer training updates, and are less prone to gibberish output caused by neural attention failures. In this paper, we combine neural HMM TTS with normalising flows for describing the highly non-Gaussian distribution of speech acoustics. The result is a powerful, fully probabilistic model of durations and acoustics that can be trained using exact maximum likelihood. Experiments show that a system based on our proposal needs fewer updates than comparable methods to produce accurate pronunciations and a subjective speech quality close to natural speech. Please see https://shivammehta25.github.io/OverFlow/ for audio examples and code.