Effect of choice of probability distribution, randomness, and search methods for alignment modeling in sequence-to-sequence text-to-speech synthesis using hard alignment

TL;DR

This paper explores how different probability distributions, randomness, and search strategies affect hard-alignment sequence-to-sequence TTS, finding deterministic search preferable and the binary Concrete distribution robust for natural alignments.

Contribution

It systematically investigates sampling methods and introduces the binary Concrete distribution for improved alignment modeling in hard-alignment TTS.

Findings

Deterministic search outperforms stochastic search in naturalness.

Binary Concrete distribution is robust with stochastic search.

Sampling method choice significantly impacts TTS alignment quality.

Abstract

Sequence-to-sequence text-to-speech (TTS) is dominated by soft-attention-based methods. Recently, hard-attention-based methods have been proposed to prevent fatal alignment errors, but their sampling method of discrete alignment is poorly investigated. This research investigates various combinations of sampling methods and probability distributions for alignment transition modeling in a hard-alignment-based sequence-to-sequence TTS method called SSNT-TTS. We clarify the common sampling methods of discrete variables including greedy search, beam search, and random sampling from a Bernoulli distribution in a more general way. Furthermore, we introduce the binary Concrete distribution to model discrete variables more properly. The results of a listening test shows that deterministic search is more preferable than stochastic search, and the binary Concrete distribution is robust with stochastic search for natural alignment transition.