Byakto Speech: Real-time long speech synthesis with convolutional neural network: Transfer learning from English to Bangla

TL;DR

This paper introduces Byakta, an open-source bilingual Bangla-English speech synthesis system utilizing transfer learning, and provides a new benchmark dataset for Bangla TTS evaluation, addressing resource limitations and script complexity.

Contribution

The work presents the first open-source bilingual TTS system for Bangla and English, along with a novel automated scoring metric and a benchmark dataset for Bangla speech synthesis.

Findings

Achieved improved speech quality for Bangla using transfer learning.

Developed a new benchmark dataset for Bangla speech synthesis evaluation.

Proposed an automated scoring metric based on speech recognition models.

Abstract

Speech synthesis is one of the challenging tasks to automate by deep learning, also being a low-resource language there are very few attempts at Bangla speech synthesis. Most of the existing works can't work with anything other than simple Bangla characters script, very short sentences, etc. This work attempts to solve these problems by introducing Byakta, the first-ever open-source deep learning-based bilingual (Bangla and English) text to a speech synthesis system. A speech recognition model-based automated scoring metric was also proposed to evaluate the performance of a TTS model. We also introduce a test benchmark dataset for Bangla speech synthesis models for evaluating speech quality. The TTS is available at https://github.com/zabir-nabil/bangla-tts