MahaTTS: A Unified Framework for Multilingual Text-to-Speech Synthesis

TL;DR

MahaTTS-v2 is a multilingual multi-speaker TTS system designed for Indic languages, leveraging semantic extraction and flow models to improve expressive speech synthesis across diverse languages.

Contribution

Introduces MahaTTS-v2, a novel multilingual TTS framework for Indic languages using Wav2Vec2.0, language models, and conditional flow models, trained on 20K hours of data.

Findings

Outperforms existing TTS frameworks in multilingual settings

Achieves high-quality expressive speech synthesis for Indic languages

Demonstrates effectiveness with extensive Indian language data

Abstract

Current Text-to-Speech models pose a multilingual challenge, where most of the models traditionally focus on English and European languages, thereby hurting the potential to provide access to information to many more people. To address this gap, we introduce MahaTTS-v2 a Multilingual Multi-speaker Text-To-Speech (TTS) system that has excellent multilingual expressive capabilities in Indic languages. The model has been trained on around 20K hours of data specifically focused on Indian languages. Our approach leverages Wav2Vec2.0 tokens for semantic extraction, and a Language Model (LM) for text-to-semantic modeling. Additionally, we have used a Conditional Flow Model (CFM) for semantics to melspectogram generation. The experimental results indicate the effectiveness of the proposed approach over other frameworks. Our code is available at https://github.com/dubverse-ai/MahaTTSv2