Assessing the Feasibility of Lightweight Whisper Models for Low-Resource Urdu Transcription

TL;DR

This paper evaluates lightweight Whisper models for Urdu speech recognition in low-resource settings, showing promising results with Whisper-Small but highlighting ongoing challenges in phonetic accuracy and lexical coherence.

Contribution

It provides the first benchmark of lightweight Whisper models on Urdu without fine-tuning, revealing their potential and limitations for low-resource ASR.

Findings

Whisper-Small achieves 33.68% WER on Urdu dataset

Lightweight models outperform larger variants in low-resource settings

Persistent phonetic and lexical challenges remain in Urdu ASR

Abstract

This study evaluates the feasibility of lightweight Whisper models (Tiny, Base, Small) for Urdu speech recognition in low-resource settings. Despite Urdu being the 10th most spoken language globally with over 230 million speakers, its representation in automatic speech recognition (ASR) systems remains limited due to dialectal diversity, code-switching, and sparse training data. We benchmark these models on a curated Urdu dataset using word error rate (WER), without fine-tuning. Results show Whisper-Small achieves the lowest error rates (33.68\% WER), outperforming Tiny (67.08\% WER) and Base (53.67\% WER). Qualitative analysis reveals persistent challenges in phonetic accuracy and lexical coherence, particularly for complex utterances. While Whisper-Small demonstrates promise for deployable Urdu ASR, significant gaps remain. Our findings emphasize lay the groundwork for future research into effective, low-resource ASR systems.