Pearl: A Multimodal Culturally-Aware Arabic Instruction Dataset

TL;DR

PEARL is a large-scale Arabic multimodal dataset and benchmark designed to improve cultural understanding in vision-language models, addressing biases and enabling nuanced cultural reasoning.

Contribution

It introduces a culturally-aware Arabic multimodal dataset with extensive annotations and evaluation benchmarks for advancing culturally-informed multimodal AI research.

Findings

Reasoning-centric instruction alignment enhances cultural grounding in models.

PEARL dataset covers ten culturally significant domains across Arab countries.

Models show improved cultural understanding with PEARL-based training.

Abstract

Mainstream large vision-language models (LVLMs) inherently encode cultural biases, highlighting the need for diverse multimodal datasets. To address this gap, we introduce PEARL, a large-scale Arabic multimodal dataset and benchmark explicitly designed for cultural understanding. Constructed through advanced agentic workflows and extensive human-in-the-loop annotations by 37 annotators from across the Arab world, PEARL comprises over 309K multimodal examples spanning ten culturally significant domains covering all Arab countries. We further provide two robust evaluation benchmarks (PEARL and PEARL-LITE) along with a specialized subset (PEARL-X) explicitly developed to assess nuanced cultural variations. Comprehensive evaluations on state-of-the-art open and proprietary LVLMs demonstrate that reasoning-centric instruction alignment substantially improves models' cultural grounding compared to conventional scaling methods. PEARL establishes a foundational resource for advancing culturally-informed multimodal modeling research. All datasets and benchmarks are publicly available.