Enhancing Word-Level Semantic Representation via Dependency Structure for Expressive Text-to-Speech Synthesis

TL;DR

This paper introduces a novel method that enhances word-level semantic representations for expressive TTS by integrating dependency structures with BERT embeddings, leading to more natural and expressive speech synthesis.

Contribution

It proposes a dependency-structure-aware enhancement of BERT embeddings using RGGN, improving semantic representation for TTS.

Findings

Improved speech naturalness and expressiveness in Mandarin and English datasets.

Enhanced semantic representation leads to better TTS performance.

The method outperforms baseline models in subjective and objective evaluations.

Abstract

Exploiting rich linguistic information in raw text is crucial for expressive text-to-speech (TTS). As large scale pre-trained text representation develops, bidirectional encoder representations from Transformers (BERT) has been proven to embody semantic information and employed to TTS recently. However, original or simply fine-tuned BERT embeddings still cannot provide sufficient semantic knowledge that expressive TTS models should take into account. In this paper, we propose a word-level semantic representation enhancing method based on dependency structure and pre-trained BERT embedding. The BERT embedding of each word is reprocessed considering its specific dependencies and related words in the sentence, to generate more effective semantic representation for TTS. To better utilize the dependency structure, relational gated graph network (RGGN) is introduced to make semantic information flow and aggregate through the dependency structure. The experimental results show that the proposed method can further improve the naturalness and expressiveness of synthesized speeches on both Mandarin and English datasets.