Beyond Orthography: Automatic Recovery of Short Vowels and Dialectal Sounds in Arabic

TL;DR

This paper introduces a new framework for recognizing dialectal and borrowed sounds in Arabic, extending beyond standard orthography, and demonstrates its effectiveness with limited data and a novel dialectal speech dataset.

Contribution

It presents a novel dialectal sound and vowelization recovery framework and introduces ArabVoice15, a unique dataset for dialectal Arabic speech recognition.

Findings

Improved character error rate by ~7% on ArabVoice15 with limited training data.

Demonstrated effectiveness of the framework across three test sets.

Provided detailed annotation guidelines and dialectal confusion analysis.

Abstract

This paper presents a novel Dialectal Sound and Vowelization Recovery framework, designed to recognize borrowed and dialectal sounds within phonologically diverse and dialect-rich languages, that extends beyond its standard orthographic sound sets. The proposed framework utilized a quantized sequence of input with(out) continuous pretrained self-supervised representation. We show the efficacy of the pipeline using limited data for Arabic, a dialect-rich language containing more than 22 major dialects. Phonetically correct transcribed speech resources for dialectal Arabic are scarce. Therefore, we introduce ArabVoice15, a first-of-its-kind, curated test set featuring 5 hours of dialectal speech across 15 Arab countries, with phonetically accurate transcriptions, including borrowed and dialect-specific sounds. We described in detail the annotation guideline along with the analysis of the dialectal confusion pairs. Our extensive evaluation includes both subjective -- human perception tests and objective measures. Our empirical results, reported with three test sets, show that with only one and half hours of training data, our model improve character error rate by ~ 7\% in ArabVoice15 compared to the baseline.