ManchuTTS: Towards High-Quality Manchu Speech Synthesis via Flow Matching and Hierarchical Text Representation

TL;DR

ManchuTTS introduces a hierarchical, flow-based speech synthesis model tailored for the endangered Manchu language, effectively addressing data scarcity and linguistic complexity to produce high-quality speech.

Contribution

This work presents the first Manchu TTS dataset, a hierarchical text representation, and a novel flow-matching Transformer model with hierarchical contrastive loss for agglutinative language synthesis.

Findings

Achieved a MOS of 4.52 with limited training data

Hierarchical guidance improves pronunciation accuracy by 31%

Prosodic naturalness increased by 27%

Abstract

As an endangered language, Manchu presents unique challenges for speech synthesis, including severe data scarcity and strong phonological agglutination. This paper proposes ManchuTTS(Manchu Text to Speech), a novel approach tailored to Manchu's linguistic characteristics. To handle agglutination, this method designs a three-tier text representation (phoneme, syllable, prosodic) and a cross-modal hierarchical attention mechanism for multi-granular alignment. The synthesis model integrates deep convolutional networks with a flow-matching Transformer, enabling efficient, non-autoregressive generation. This method further introduce a hierarchical contrastive loss to guide structured acoustic-linguistic correspondence. To address low-resource constraints, This method construct the first Manchu TTS dataset and employ a data augmentation strategy. Experiments demonstrate that ManchuTTS attains a MOS of 4.52 using a 5.2-hour training subset derived from our full 6.24-hour annotated corpus, outperforming all baseline models by a notable margin. Ablations confirm hierarchical guidance improves agglutinative word pronunciation accuracy (AWPA) by 31% and prosodic naturalness by 27%.