Optimizing RAG Pipelines for Arabic: A Systematic Analysis of Core Components

TL;DR

This paper systematically evaluates and optimizes RAG pipeline components for Arabic, identifying best strategies for chunking, embedding, reranking, and generation to improve performance across various datasets.

Contribution

It provides the first comprehensive empirical analysis of RAG components tailored for Arabic, offering practical guidelines for building effective Arabic RAG systems.

Findings

Sentence-aware chunking outperforms other segmentation methods.

BGE-M3 and Multilingual-E5-large are the most effective embedding models.

Reranker significantly improves answer faithfulness in complex datasets.

Abstract

Retrieval-Augmented Generation (RAG) has emerged as a powerful architecture for combining the precision of retrieval systems with the fluency of large language models. While several studies have investigated RAG pipelines for high-resource languages, the optimization of RAG components for Arabic remains underexplored. This study presents a comprehensive empirical evaluation of state-of-the-art RAG components-including chunking strategies, embedding models, rerankers, and language models-across a diverse set of Arabic datasets. Using the RAGAS framework, we systematically compare performance across four core metrics: context precision, context recall, answer faithfulness, and answer relevancy. Our experiments demonstrate that sentence-aware chunking outperforms all other segmentation methods, while BGE-M3 and Multilingual-E5-large emerge as the most effective embedding models. The inclusion of a reranker (bge-reranker-v2-m3) significantly boosts faithfulness in complex datasets, and Aya-8B surpasses StableLM in generation quality. These findings provide critical insights for building high-quality Arabic RAG pipelines and offer practical guidelines for selecting optimal components across different document types.