Low-Resource End-to-end Sanskrit TTS using Tacotron2, WaveGlow and Transfer Learning

TL;DR

This paper explores fine-tuning English-pretrained Tacotron2 and WaveGlow models with limited Sanskrit data to develop a low-resource end-to-end Sanskrit TTS system, achieving promising speech naturalness.

Contribution

It demonstrates the effectiveness of transfer learning for Sanskrit TTS using limited data, addressing resource scarcity challenges.

Findings

Achieved MOS of 3.38 with 37 evaluators.

Used only 2.5 hours of Sanskrit speech data.

Showed successful transfer learning from English to Sanskrit.

Abstract

End-to-end text-to-speech (TTS) systems have been developed for European languages like English and Spanish with state-of-the-art speech quality, prosody, and naturalness. However, development of end-to-end TTS for Indian languages is lagging behind in terms of quality. The challenges involved in such a task are: 1) scarcity of quality training data; 2) low efficiency during training and inference; 3) slow convergence in the case of large vocabulary size. In our work reported in this paper, we have investigated the use of fine-tuning the English-pretrained Tacotron2 model with limited Sanskrit data to synthesize natural sounding speech in Sanskrit in low resource settings. Our experiments show encouraging results, achieving an overall MOS of 3.38 from 37 evaluators with good Sanskrit spoken knowledge. This is really a very good result, considering the fact that the speech data we have used is of duration 2.5 hours only.