Single-stage TTS with Masked Audio Token Modeling and Semantic Knowledge Distillation

TL;DR

This paper introduces a single-stage text-to-speech model using masked audio token modeling and semantic knowledge distillation, achieving high-quality speech synthesis with a more efficient architecture.

Contribution

The paper presents a novel single-stage TTS approach that narrows the quality gap with two-stage systems through semantic knowledge distillation.

Findings

Improved speech quality, intelligibility, and speaker similarity over baseline

Single-stage model approaches two-stage system performance in quality

Model offers a more compact and efficient TTS architecture

Abstract

Audio token modeling has become a powerful framework for speech synthesis, with two-stage approaches employing semantic tokens remaining prevalent. In this paper, we aim to simplify this process by introducing a semantic knowledge distillation method that enables high-quality speech generation in a single stage. Our proposed model improves speech quality, intelligibility, and speaker similarity compared to a single-stage baseline. Although two-stage systems still lead in intelligibility, our model significantly narrows the gap while delivering comparable speech quality. These findings showcase the potential of single-stage models to achieve efficient, high-quality TTS with a more compact and streamlined architecture.