Empowering Low-Resource Language ASR via Large-Scale Pseudo Labeling

TL;DR

This paper presents a large-scale pseudo-labeling framework to improve automatic speech recognition for low-resource languages like Hindi, leveraging multiple models and a new YouTube-based benchmark.

Contribution

It introduces a generic pseudo-labeling framework combining multiple models and evaluators, validated on a new diverse YouTube audio benchmark for Hindi ASR.

Findings

Pseudo-labeled data from YouTube improves Hindi ASR performance.

Augmentation with pseudo labels does not harm out-of-domain performance.

The approach is validated on the new IndicYT benchmark.

Abstract

In this study, we tackle the challenge of limited labeled data for low-resource languages in ASR, focusing on Hindi. Specifically, we explore pseudo-labeling, by proposing a generic framework combining multiple ideas from existing works. Our framework integrates multiple base models for transcription and evaluators for assessing audio-transcript pairs, resulting in robust pseudo-labeling for low resource languages. We validate our approach with a new benchmark, IndicYT, comprising diverse YouTube audio files from multiple content categories. Our findings show that augmenting pseudo labeled data from YouTube with existing training data leads to significant performance improvements on IndicYT, without affecting performance on out-of-domain benchmarks, demonstrating the efficacy of pseudo-labeled data in enhancing ASR capabilities for low-resource languages. The benchmark, code and models developed as a part of this work will be made publicly available.