GSA-TTS : Toward Zero-Shot Speech Synthesis based on Gradual Style Adaptor

TL;DR

GSA-TTS introduces a hierarchical style encoder for zero-shot speech synthesis, capturing local and global speaking styles from acoustic references to improve naturalness, similarity, and controllability.

Contribution

It proposes a novel gradual style adaptor with hierarchical encoding for zero-shot speech synthesis, enhancing style representation and interpretability.

Findings

Achieves promising naturalness and speaker similarity on unseen speakers.

Demonstrates improved intelligibility and style controllability.

Provides insights into interpretability of style encoding.

Abstract

We present the gradual style adaptor TTS (GSA-TTS) with a novel style encoder that gradually encodes speaking styles from an acoustic reference for zero-shot speech synthesis. GSA first captures the local style of each semantic sound unit. Then the local styles are combined by self-attention to obtain a global style condition. This semantic and hierarchical encoding strategy provides a robust and rich style representation for an acoustic model. We test GSA-TTS on unseen speakers and obtain promising results regarding naturalness, speaker similarity, and intelligibility. Additionally, we explore the potential of GSA in terms of interpretability and controllability, which stems from its hierarchical structure.