HiGNN-TTS: Hierarchical Prosody Modeling with Graph Neural Networks for Expressive Long-form TTS

TL;DR

HiGNN-TTS introduces a hierarchical graph neural network approach with global nodes and contextual attention to enhance prosody modeling, enabling more natural and expressive long-form speech synthesis.

Contribution

The paper proposes a novel hierarchical GNN framework with global nodes and attention mechanisms for improved prosody in long-form TTS.

Findings

Significant improvement in speech naturalness and expressiveness.

Effective hierarchical prosody learning demonstrated through ablation studies.

Enhanced inter-sentence prosody modeling with contextual attention.

Abstract

Recent advances in text-to-speech, particularly those based on Graph Neural Networks (GNNs), have significantly improved the expressiveness of short-form synthetic speech. However, generating human-parity long-form speech with high dynamic prosodic variations is still challenging. To address this problem, we expand the capabilities of GNNs with a hierarchical prosody modeling approach, named HiGNN-TTS. Specifically, we add a virtual global node in the graph to strengthen the interconnection of word nodes and introduce a contextual attention mechanism to broaden the prosody modeling scope of GNNs from intra-sentence to inter-sentence. Additionally, we perform hierarchical supervision from acoustic prosody on each node of the graph to capture the prosodic variations with a high dynamic range. Ablation studies show the effectiveness of HiGNN-TTS in learning hierarchical prosody. Both objective and subjective evaluations demonstrate that HiGNN-TTS significantly improves the naturalness and expressiveness of long-form synthetic speech.