Lightweight End-to-end Text-to-speech Synthesis for low resource on-device applications

TL;DR

This paper introduces LE2E, a lightweight end-to-end text-to-speech model that delivers high-quality speech with significantly reduced computational resources, suitable for low-resource on-device applications.

Contribution

The paper presents a novel lightweight E2E-TTS model that outperforms existing models in size and speed while maintaining high speech quality, and demonstrates the advantages of end-to-end training over two-stage methods.

Findings

Achieves state-of-the-art performance on LJSpeech dataset.

Model is up to 90% smaller and 10x faster in real-time factor.

End-to-end training yields better quality than two-stage approaches.

Abstract

Recent works have shown that modelling raw waveform directly from text in an end-to-end (E2E) fashion produces more natural-sounding speech than traditional neural text-to-speech (TTS) systems based on a cascade or two-stage approach. However, current E2E state-of-the-art models are computationally complex and memory-consuming, making them unsuitable for real-time offline on-device applications in low-resource scenarios. To address this issue, we propose a Lightweight E2E-TTS (LE2E) model that generates high-quality speech requiring minimal computational resources. We evaluate the proposed model on the LJSpeech dataset and show that it achieves state-of-the-art performance while being up to $90\%$ smaller in terms of model parameters and $10\times$ faster in real-time-factor. Furthermore, we demonstrate that the proposed E2E training paradigm achieves better quality compared to an equivalent architecture trained in a two-stage approach. Our results suggest that LE2E is a promising approach for developing real-time, high quality, low-resource TTS applications for on-device applications.