FastGraphTTS: An Ultrafast Syntax-Aware Speech Synthesis Framework

TL;DR

FastGraphTTS introduces a syntax-aware speech synthesis framework that leverages dependency graphs and graph encoding to improve prosody and efficiency in text-to-speech systems across multiple languages and speaker scenarios.

Contribution

The paper presents a novel integration of graph-to-sequence modeling into end-to-end TTS, achieving enhanced prosody and 5x acceleration through specialized AI chip operators.

Findings

Improved prosodic consistency and subjective quality in speech synthesis.

Demonstrated effectiveness across English and Mandarin datasets.

Achieved 5x faster inference with optimized hardware operators.

Abstract

This paper integrates graph-to-sequence into an end-to-end text-to-speech framework for syntax-aware modelling with syntactic information of input text. Specifically, the input text is parsed by a dependency parsing module to form a syntactic graph. The syntactic graph is then encoded by a graph encoder to extract the syntactic hidden information, which is concatenated with phoneme embedding and input to the alignment and flow-based decoding modules to generate the raw audio waveform. The model is experimented on two languages, English and Mandarin, using single-speaker, few samples of target speakers, and multi-speaker datasets, respectively. Experimental results show better prosodic consistency performance between input text and generated audio, and also get higher scores in the subjective prosodic evaluation, and show the ability of voice conversion. Besides, the efficiency of the model is largely boosted through the design of the AI chip operator with 5x acceleration.