Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis

TL;DR

This paper introduces global style tokens (GSTs), a novel unsupervised embedding approach within Tacotron that models, controls, and transfers speech style and expressiveness without explicit labels.

Contribution

The work presents GSTs as a new method for unsupervised style modeling in end-to-end speech synthesis, enabling style control and transfer.

Findings

GSTs effectively model a wide range of acoustic styles

They enable independent control of speaking style and speed

GSTs can transfer styles from single clips to large text corpora

Abstract

In this work, we propose "global style tokens" (GSTs), a bank of embeddings that are jointly trained within Tacotron, a state-of-the-art end-to-end speech synthesis system. The embeddings are trained with no explicit labels, yet learn to model a large range of acoustic expressiveness. GSTs lead to a rich set of significant results. The soft interpretable "labels" they generate can be used to control synthesis in novel ways, such as varying speed and speaking style - independently of the text content. They can also be used for style transfer, replicating the speaking style of a single audio clip across an entire long-form text corpus. When trained on noisy, unlabeled found data, GSTs learn to factorize noise and speaker identity, providing a path towards highly scalable but robust speech synthesis.