Query Resolution for Conversational Search with Limited Supervision

TL;DR

This paper introduces QuReTeC, a neural model for query resolution in conversational search that uses limited supervision and improves multi-turn passage retrieval by effectively incorporating conversational context.

Contribution

The paper presents a novel neural query resolution model and a distant supervision method that reduces the need for human-labeled data in conversational search tasks.

Findings

QuReTeC outperforms existing models in query resolution accuracy.

Distant supervision effectively reduces reliance on human annotations.

Incorporating QuReTeC improves retrieval performance on TREC CAsT dataset.

Abstract

In this work we focus on multi-turn passage retrieval as a crucial component of conversational search. One of the key challenges in multi-turn passage retrieval comes from the fact that the current turn query is often underspecified due to zero anaphora, topic change, or topic return. Context from the conversational history can be used to arrive at a better expression of the current turn query, defined as the task of query resolution. In this paper, we model the query resolution task as a binary term classification problem: for each term appearing in the previous turns of the conversation decide whether to add it to the current turn query or not. We propose QuReTeC (Query Resolution by Term Classification), a neural query resolution model based on bidirectional transformers. We propose a distant supervision method to automatically generate training data by using query-passage relevance labels. Such labels are often readily available in a collection either as human annotations or inferred from user interactions. We show that QuReTeC outperforms state-of-the-art models, and furthermore, that our distant supervision method can be used to substantially reduce the amount of human-curated data required to train QuReTeC. We incorporate QuReTeC in a multi-turn, multi-stage passage retrieval architecture and demonstrate its effectiveness on the TREC CAsT dataset.