PCoQA: Persian Conversational Question Answering Dataset

TL;DR

The paper introduces PCoQA, a Persian conversational question answering dataset with over 9,000 dialogs, designed to challenge models with open-ended answers and longer responses, and evaluates baseline and pre-trained models on it.

Contribution

This work presents the first Persian conversational QA dataset, PCoQA, along with benchmark results, enabling research in Persian language understanding and dialogue systems.

Findings

Baseline models perform modestly on PCoQA.

Pre-trained models improve performance significantly.

The dataset presents challenges like open-ended answers and longer responses.

Abstract

Humans seek information regarding a specific topic through performing a conversation containing a series of questions and answers. In the pursuit of conversational question answering research, we introduce the PCoQA, the first \textbf{P}ersian \textbf{Co}nversational \textbf{Q}uestion \textbf{A}nswering dataset, a resource comprising information-seeking dialogs encompassing a total of 9,026 contextually-driven questions. Each dialog involves a questioner, a responder, and a document from the Wikipedia; The questioner asks several inter-connected questions from the text and the responder provides a span of the document as the answer for each question. PCoQA is designed to present novel challenges compared to previous question answering datasets including having more open-ended non-factual answers, longer answers, and fewer lexical overlaps. This paper not only presents the comprehensive PCoQA dataset but also reports the performance of various benchmark models. Our models include baseline models and pre-trained models, which are leveraged to boost the performance of the model. The dataset and benchmarks are available at our Github page.