Alternate Endings: Improving Prosody for Incremental Neural TTS with Predicted Future Text Input

TL;DR

This paper explores how predicted future text input can improve prosody in incremental neural TTS, showing that language model predictions enhance naturalness compared to no lookahead, with slight advantages over random predictions.

Contribution

It demonstrates that using predicted future text in incremental TTS improves prosody, offering a practical method to enhance speech naturalness when full context isn't available.

Findings

Predicted future text improves prosody over zero-word lookahead.

Language model predictions outperform random predictions.

Perceptual tests confirm prosodic improvements.

Abstract

The prosody of a spoken word is determined by its surrounding context. In incremental text-to-speech synthesis, where the synthesizer produces an output before it has access to the complete input, the full context is often unknown which can result in a loss of naturalness in the synthesized speech. In this paper, we investigate whether the use of predicted future text can attenuate this loss. We compare several test conditions of next future word: (a) unknown (zero-word), (b) language model predicted, (c) randomly predicted and (d) ground-truth. We measure the prosodic features (pitch, energy and duration) and find that predicted text provides significant improvements over a zero-word lookahead, but only slight gains over random-word lookahead. We confirm these results with a perceptive test.