One TTS Alignment To Rule Them All

TL;DR

This paper introduces a universal alignment learning framework for neural TTS models that enhances alignment robustness, convergence speed, and speech quality across various architectures, including autoregressive and non-autoregressive models.

Contribution

It adapts the RAD-TTS alignment mechanism with a combined forward-sum and Viterbi algorithm, improving TTS model performance and robustness.

Findings

Improved alignment convergence speed.

Enhanced robustness to long utterance errors.

Better perceived speech quality by human evaluators.

Abstract

Speech-to-text alignment is a critical component of neural textto-speech (TTS) models. Autoregressive TTS models typically use an attention mechanism to learn these alignments on-line. However, these alignments tend to be brittle and often fail to generalize to long utterances and out-of-domain text, leading to missing or repeating words. Most non-autoregressive endto-end TTS models rely on durations extracted from external sources. In this paper we leverage the alignment mechanism proposed in RAD-TTS as a generic alignment learning framework, easily applicable to a variety of neural TTS models. The framework combines forward-sum algorithm, the Viterbi algorithm, and a simple and efficient static prior. In our experiments, the alignment learning framework improves all tested TTS architectures, both autoregressive (Flowtron, Tacotron 2) and non-autoregressive (FastPitch, FastSpeech 2, RAD-TTS). Specifically, it improves alignment convergence speed of existing attention-based mechanisms, simplifies the training pipeline, and makes the models more robust to errors on long utterances. Most importantly, the framework improves the perceived speech synthesis quality, as judged by human evaluators.