Few-Shot Prompting for Extractive Quranic QA with Instruction-Tuned LLMs

TL;DR

This paper demonstrates that few-shot prompting with instruction-tuned large language models significantly improves extractive Quranic question answering, especially in low-resource, semantically complex contexts.

Contribution

It introduces a specialized Arabic prompt framework and a post-processing system, advancing extractive QA performance with instruction-tuned LLMs for Quranic texts.

Findings

Large language models outperform traditional fine-tuned models.

The best configuration achieves a pAP10 score of 0.637.

Prompt-based instruction tuning is effective for low-resource, semantically rich QA tasks.

Abstract

This paper presents two effective approaches for Extractive Question Answering (QA) on the Quran. It addresses challenges related to complex language, unique terminology, and deep meaning in the text. The second uses few-shot prompting with instruction-tuned large language models such as Gemini and DeepSeek. A specialized Arabic prompt framework is developed for span extraction. A strong post-processing system integrates subword alignment, overlap suppression, and semantic filtering. This improves precision and reduces hallucinations. Evaluations show that large language models with Arabic instructions outperform traditional fine-tuned models. The best configuration achieves a pAP10 score of 0.637. The results confirm that prompt-based instruction tuning is effective for low-resource, semantically rich QA tasks.