ParaQA: A Question Answering Dataset with Paraphrase Responses for Single-Turn Conversation

TL;DR

ParaQA is a new dataset for single-turn conversational question answering over knowledge graphs, featuring multiple paraphrased answers per question to improve answer diversity and robustness.

Contribution

The paper introduces ParaQA, a dataset with multiple paraphrased responses per question, created via semi-automated back-translation techniques, filling a gap in existing QA datasets.

Findings

Baseline models demonstrate the dataset's utility in improving answer diversity.

Multiple paraphrases enhance the robustness of QA systems.

The dataset is publicly available for research use.

Abstract

This paper presents ParaQA, a question answering (QA) dataset with multiple paraphrased responses for single-turn conversation over knowledge graphs (KG). The dataset was created using a semi-automated framework for generating diverse paraphrasing of the answers using techniques such as back-translation. The existing datasets for conversational question answering over KGs (single-turn/multi-turn) focus on question paraphrasing and provide only up to one answer verbalization. However, ParaQA contains 5000 question-answer pairs with a minimum of two and a maximum of eight unique paraphrased responses for each question. We complement the dataset with baseline models and illustrate the advantage of having multiple paraphrased answers through commonly used metrics such as BLEU and METEOR. The ParaQA dataset is publicly available on a persistent URI for broader usage and adaptation in the research community.