Enabling Beam Search for Language Model-Based Text-to-Speech Synthesis

TL;DR

This paper introduces TRAD-BS, a maximisation-based decoding method for language model-based TTS that reduces artefacts and mispronunciations, improving speech quality and speaker consistency.

Contribution

It proposes a novel beam search variant, TRAD-BS, tailored for TTS to enhance decoding quality and address issues caused by sampling strategies.

Findings

Fewer mispronunciations in generated speech.

Improved speaker consistency.

Enhanced speech naturalness.

Abstract

Tokenising continuous speech into sequences of discrete tokens and modelling them with language models (LMs) has led to significant success in text-to-speech (TTS) synthesis. Although these models can generate speech with high quality and naturalness, their synthesised samples can still suffer from artefacts, mispronunciation, word repeating, etc. In this paper, we argue these undesirable properties could partly be caused by the randomness of sampling-based strategies during the autoregressive decoding of LMs. Therefore, we look at maximisation-based decoding approaches and propose Temporal Repetition Aware Diverse Beam Search (TRAD-BS) to find the most probable sequences of the generated speech tokens. Experiments with two state-of-the-art LM-based TTS models demonstrate that our proposed maximisation-based decoding strategy generates speech with fewer mispronunciations and improved speaker consistency.