Open-Retrieval Conversational Question Answering

TL;DR

This paper introduces ORConvQA, a new open-retrieval conversational question answering framework with a dedicated dataset, demonstrating the importance of retrieval and history modeling for improved conversational search performance.

Contribution

It presents the ORConvQA setting, a new dataset OR-QuAC, and an end-to-end Transformer-based system, emphasizing the role of learnable retrieval and history modeling in conversational QA.

Findings

Learnable retriever is essential for ORConvQA.

History modeling improves system performance.

Reranker adds regularization benefits.

Abstract

Conversational search is one of the ultimate goals of information retrieval. Recent research approaches conversational search by simplified settings of response ranking and conversational question answering, where an answer is either selected from a given candidate set or extracted from a given passage. These simplifications neglect the fundamental role of retrieval in conversational search. To address this limitation, we introduce an open-retrieval conversational question answering (ORConvQA) setting, where we learn to retrieve evidence from a large collection before extracting answers, as a further step towards building functional conversational search systems. We create a dataset, OR-QuAC, to facilitate research on ORConvQA. We build an end-to-end system for ORConvQA, featuring a retriever, a reranker, and a reader that are all based on Transformers. Our extensive experiments on OR-QuAC demonstrate that a learnable retriever is crucial for ORConvQA. We further show that our system can make a substantial improvement when we enable history modeling in all system components. Moreover, we show that the reranker component contributes to the model performance by providing a regularization effect. Finally, further in-depth analyses are performed to provide new insights into ORConvQA.