PAQA: Toward ProActive Open-Retrieval Question Answering

TL;DR

This paper introduces PAQA, an extension to an existing dataset, to improve conversational search systems by generating relevant clarifying questions that address ambiguities in user queries and documents.

Contribution

The work extends the AmbiNQ dataset with clarifying questions and evaluates models on how passage retrieval influences ambiguity detection and question generation.

Findings

PAQA enhances dataset for clarifying questions in conversational search.

Passage retrieval significantly impacts ambiguity detection accuracy.

Models trained on PAQA show improved clarification question generation.

Abstract

Conversational systems have made significant progress in generating natural language responses. However, their potential as conversational search systems is currently limited due to their passive role in the information-seeking process. One major limitation is the scarcity of datasets that provide labelled ambiguous questions along with a supporting corpus of documents and relevant clarifying questions. This work aims to tackle the challenge of generating relevant clarifying questions by taking into account the inherent ambiguities present in both user queries and documents. To achieve this, we propose PAQA, an extension to the existing AmbiNQ dataset, incorporating clarifying questions. We then evaluate various models and assess how passage retrieval impacts ambiguity detection and the generation of clarifying questions. By addressing this gap in conversational search systems, we aim to provide additional supervision to enhance their active participation in the information-seeking process and provide users with more accurate results.