Self-supervised Context-aware Style Representation for Expressive Speech Synthesis

TL;DR

This paper introduces a self-supervised framework for learning style representations from plain text for expressive speech synthesis, improving naturalness and emotional transitions without relying on costly labeled data.

Contribution

It proposes a novel self-supervised approach using contrastive learning and deep clustering to derive style embeddings from text, integrated into a multi-style Transformer TTS system.

Findings

Enhanced subjective quality in synthesized speech

More natural emotion transitions in long paragraphs

Effective style representation without labeled data

Abstract

Expressive speech synthesis, like audiobook synthesis, is still challenging for style representation learning and prediction. Deriving from reference audio or predicting style tags from text requires a huge amount of labeled data, which is costly to acquire and difficult to define and annotate accurately. In this paper, we propose a novel framework for learning style representation from abundant plain text in a self-supervised manner. It leverages an emotion lexicon and uses contrastive learning and deep clustering. We further integrate the style representation as a conditioned embedding in a multi-style Transformer TTS. Comparing with multi-style TTS by predicting style tags trained on the same dataset but with human annotations, our method achieves improved results according to subjective evaluations on both in-domain and out-of-domain test sets in audiobook speech. Moreover, with implicit context-aware style representation, the emotion transition of synthesized audio in a long paragraph appears more natural. The audio samples are available on the demo web.