PQuAD: A Persian Question Answering Dataset

TL;DR

PQuAD is a large, diverse Persian reading comprehension dataset with answerable and unanswerable questions, designed to advance Persian question answering systems and benchmark their performance.

Contribution

This paper introduces PQuAD, the first large-scale Persian QA dataset with 80,000 questions, including adversarial unanswerable ones, facilitating research and development in Persian NLP.

Findings

State-of-the-art models achieve 74.8% EM and 87.6% F1 on PQuAD.

The dataset exhibits significant diversity and difficulty for Persian QA tasks.

PQuAD serves as a benchmark for future Persian question answering research.

Abstract

We present Persian Question Answering Dataset (PQuAD), a crowdsourced reading comprehension dataset on Persian Wikipedia articles. It includes 80,000 questions along with their answers, with 25% of the questions being adversarially unanswerable. We examine various properties of the dataset to show the diversity and the level of its difficulty as an MRC benchmark. By releasing this dataset, we aim to ease research on Persian reading comprehension and development of Persian question answering systems. Our experiments on different state-of-the-art pre-trained contextualized language models show 74.8% Exact Match (EM) and 87.6% F1-score that can be used as the baseline results for further research on Persian QA.