Efficient ASR for Low-Resource Languages: Leveraging Cross-Lingual Unlabeled Data

TL;DR

This paper demonstrates that strategic cross-lingual pretraining on unlabeled speech data can significantly improve ASR performance for low-resource languages, reducing the need for large models and labeled datasets.

Contribution

It introduces a scalable pretraining approach using unlabeled data and morphologically-aware tokenization to develop effective ASR models for low-resource languages.

Findings

Achieved comparable performance with 300M model versus larger models.

Outperformed Whisper Large v3 on Persian ASR.

Showed data relevance and pretraining strategy are crucial for low-resource ASR.

Abstract

Automatic speech recognition for low-resource languages remains fundamentally constrained by the scarcity of labeled data and computational resources required by state-of-the-art models. We present a systematic investigation into cross-lingual continuous pretraining for low-resource languages, using Perso-Arabic languages (Persian, Arabic, and Urdu) as our primary case study. Our approach demonstrates that strategic utilization of unlabeled speech data can effectively bridge the resource gap without sacrificing recognition accuracy. We construct a 3,000-hour multilingual corpus through a scalable unlabeled data collection pipeline and employ targeted continual pretraining combined with morphologically-aware tokenization to develop a 300M parameter model that achieves performance comparable to systems 5 times larger. Our model outperforms Whisper Large v3 (1.5B parameters) on Persian and achieves competitive results on Arabic and Urdu despite using significantly fewer parameters and substantially less labeled data. These findings challenge the prevailing assumption that ASR quality scales primarily with model size, revealing instead that data relevance and strategic pretraining are more critical factors for low-resource scenarios. This work provides a practical pathway toward inclusive speech technology, enabling effective ASR for underrepresented languages without dependence on massive computational infrastructure or proprietary datasets.