BnTTS: Few-Shot Speaker Adaptation in Low-Resource Setting

TL;DR

This paper presents BnTTS, a novel few-shot speaker adaptation framework for Bangla TTS that effectively synthesizes natural and intelligible speech with minimal training data, addressing resource scarcity in low-resource language speech synthesis.

Contribution

Introduces BnTTS, the first Bangla speaker adaptation TTS system utilizing minimal data, integrating Bangla into a multilingual TTS pipeline based on XTTS architecture.

Findings

BnTTS significantly improves speech naturalness and clarity.

Outperforms existing Bangla TTS systems in subjective evaluations.

Effective in zero-shot and few-shot adaptation scenarios.

Abstract

This paper introduces BnTTS (Bangla Text-To-Speech), the first framework for Bangla speaker adaptation-based TTS, designed to bridge the gap in Bangla speech synthesis using minimal training data. Building upon the XTTS architecture, our approach integrates Bangla into a multilingual TTS pipeline, with modifications to account for the phonetic and linguistic characteristics of the language. We pre-train BnTTS on 3.85k hours of Bangla speech dataset with corresponding text labels and evaluate performance in both zero-shot and few-shot settings on our proposed test dataset. Empirical evaluations in few-shot settings show that BnTTS significantly improves the naturalness, intelligibility, and speaker fidelity of synthesized Bangla speech. Compared to state-of-the-art Bangla TTS systems, BnTTS exhibits superior performance in Subjective Mean Opinion Score (SMOS), Naturalness, and Clarity metrics.