Custom Data Augmentation for low resource ASR using Bark and Retrieval-Based Voice Conversion

TL;DR

This paper introduces two novel methods using Bark and Retrieval-Based Voice Conversion to create customized datasets for low-resource ASR, improving data quality and enabling personalized voice synthesis.

Contribution

It presents innovative methodologies leveraging Bark and RVC for constructing tailored datasets for low-resource languages like Hindi.

Findings

Enhanced dataset quality for low-resource ASR

Improved performance of ASR models with custom datasets

Potential for high-quality personalized voice generation

Abstract

This paper proposes two innovative methodologies to construct customized Common Voice datasets for low-resource languages like Hindi. The first methodology leverages Bark, a transformer-based text-to-audio model developed by Suno, and incorporates Meta's enCodec and a pre-trained HuBert model to enhance Bark's performance. The second methodology employs Retrieval-Based Voice Conversion (RVC) and uses the Ozen toolkit for data preparation. Both methodologies contribute to the advancement of ASR technology and offer valuable insights into addressing the challenges of constructing customized Common Voice datasets for under-resourced languages. Furthermore, they provide a pathway to achieving high-quality, personalized voice generation for a range of applications.