Bengali-Loop: Community Benchmarks for Long-Form Bangla ASR and Speaker Diarization

TL;DR

Bengali-Loop introduces community benchmarks for long-form Bangla speech recognition and speaker diarization, providing datasets, evaluation protocols, and baselines to advance research in under-resourced Bengali language technology.

Contribution

This work presents the first large-scale, reproducible benchmarks for long-form Bangla ASR and speaker diarization, including datasets, annotation standards, and baseline results.

Findings

ASR baseline WER: 34.07%

Speaker diarization DER: 40.08%

Provides standardized evaluation protocols

Abstract

Bengali (Bangla) remains under-resourced in long-form speech technology despite its wide use. We present Bengali-Loop, two community benchmarks to address this gap: (1) a long-form ASR corpus of 191 recordings (158.6 hours, 792k words) from 11 YouTube channels, collected via a reproducible subtitle-extraction pipeline and human-in-the-loop transcript verification; and (2) a speaker diarization corpus of 24 recordings (22 hours, 5,744 annotated segments) with fully manual speaker-turn labels in CSV format. Both benchmarks target realistic multi-speaker, long-duration content (e.g., Bangla drama/natok). We establish baselines (Tugstugi: 34.07% WER; pyannote.audio: 40.08% DER) and provide standardized evaluation protocols (WER/CER, DER), annotation rules, and data formats to support reproducible benchmarking and future model development for Bangla long-form ASR and diarization.