Nix-TTS: Lightweight and End-to-End Text-to-Speech via Module-wise Distillation

TL;DR

Nix-TTS is a lightweight, non-autoregressive, end-to-end text-to-speech model achieved through module-wise knowledge distillation, significantly reducing size and inference time while maintaining quality.

Contribution

The paper introduces module-wise distillation for TTS, enabling independent and flexible training of encoder and decoder, resulting in a compact and efficient model.

Findings

Model size reduced by up to 89.34%

Achieves 3.04x and 8.36x speedup on CPU and Raspberry Pi

Retains naturalness and intelligibility of speech

Abstract

Several solutions for lightweight TTS have shown promising results. Still, they either rely on a hand-crafted design that reaches non-optimum size or use a neural architecture search but often suffer training costs. We present Nix-TTS, a lightweight TTS achieved via knowledge distillation to a high-quality yet large-sized, non-autoregressive, and end-to-end (vocoder-free) TTS teacher model. Specifically, we offer module-wise distillation, enabling flexible and independent distillation to the encoder and decoder module. The resulting Nix-TTS inherited the advantageous properties of being non-autoregressive and end-to-end from the teacher, yet significantly smaller in size, with only 5.23M parameters or up to 89.34% reduction of the teacher model; it also achieves over 3.04x and 8.36x inference speedup on Intel-i7 CPU and Raspberry Pi 3B respectively and still retains a fair voice naturalness and intelligibility compared to the teacher model. We provide pretrained models and audio samples of Nix-TTS.