Ensemble prosody prediction for expressive speech synthesis

TL;DR

This paper explores ensemble learning for prosody prediction in expressive speech synthesis, showing that small ensembles with a variance-based selection criterion outperform individual models.

Contribution

It introduces a novel, simple ensemble approach with a variance-based model selection criterion for improved prosody prediction in TTS.

Findings

Small ensembles provide useful diversity in prosody prediction.

The proposed variance-based selection outperforms individual models.

Ensemble methods enhance expressive speech synthesis quality.

Abstract

Generating expressive speech with rich and varied prosody continues to be a challenge for Text-to-Speech. Most efforts have focused on sophisticated neural architectures intended to better model the data distribution. Yet, in evaluations it is generally found that no single model is preferred for all input texts. This suggests an approach that has rarely been used before for Text-to-Speech: an ensemble of models. We apply ensemble learning to prosody prediction. We construct simple ensembles of prosody predictors by varying either model architecture or model parameter values. To automatically select amongst the models in the ensemble when performing Text-to-Speech, we propose a novel, and computationally trivial, variance-based criterion. We demonstrate that even a small ensemble of prosody predictors yields useful diversity, which, combined with the proposed selection criterion, outperforms any individual model from the ensemble.