Alexandria: A Multi-Domain Dialectal Arabic Machine Translation Dataset for Culturally Inclusive and Linguistically Diverse LLMs

TL;DR

Alexandria is a comprehensive, multi-domain dialectal Arabic dataset designed to improve machine translation and LLM performance across diverse Arabic dialects with detailed metadata and evaluation benchmarks.

Contribution

It introduces a large-scale, community-driven dataset with city-level dialect metadata and gender annotations, filling a gap in resources for dialectal Arabic translation.

Findings

Current LLMs show significant challenges in translating diverse Arabic dialects.

The dataset enables detailed evaluation of dialectal and gender-conditioned translation performance.

Benchmark results highlight persistent gaps in dialectal Arabic machine translation capabilities.

Abstract

Arabic is a highly diglossic language where most daily communication occurs in regional dialects rather than Modern Standard Arabic (MSA). Despite this, machine translation (MT) systems often generalize poorly to dialectal input, limiting their utility for millions of speakers. We introduce Alexandria, a large-scale, community-driven, human-translated dataset designed to bridge this gap. Alexandria covers 13 Arab countries and 11 high-impact domains, including health, education, and agriculture. Unlike previous resources, Alexandria provides unprecedented granularity by associating contributions with city-of-origin metadata, capturing authentic local varieties beyond coarse regional labels. The dataset consists of parallel English-Dialectal Arabic multi-turn conversational scenarios annotated with speaker-addressee gender configurations, enabling the study of gender-conditioned variation in dialectal use. Comprising 107K total turns, Alexandria serves as both a training resource and as a rigorous benchmark for evaluating MT and Large Language Models (LLMs). Our automatic and human evaluation benchmarks the current capabilities of Arabic-aware LLMs in translating across diverse Arabic dialects and sub-dialects while exposing significant persistent challenges. The Alexandria dataset, the creation prompts, the translation and revision guidelines, and the evaluation code are publicly available in the following repository: https://github.com/UBC-NLP/Alexandria