Learning to Relate to Previous Turns in Conversational Search

TL;DR

This paper introduces a novel method for selecting relevant historical queries in conversational search, improving retrieval effectiveness by using pseudo-labeling and multi-task learning to better incorporate conversation context.

Contribution

It proposes a new query selection method using pseudo-labeling and multi-task learning to enhance conversational search performance.

Findings

Significant improvement over baseline methods in retrieval effectiveness

Effective use of pseudo-labeling to address lack of labeled data

Broad applicability demonstrated across four datasets

Abstract

Conversational search allows a user to interact with a search system in multiple turns. A query is strongly dependent on the conversation context. An effective way to improve retrieval effectiveness is to expand the current query with historical queries. However, not all the previous queries are related to, and useful for expanding the current query. In this paper, we propose a new method to select relevant historical queries that are useful for the current query. To cope with the lack of labeled training data, we use a pseudo-labeling approach to annotate useful historical queries based on their impact on the retrieval results. The pseudo-labeled data are used to train a selection model. We further propose a multi-task learning framework to jointly train the selector and the retriever during fine-tuning, allowing us to mitigate the possible inconsistency between the pseudo labels and the changed retriever. Extensive experiments on four conversational search datasets demonstrate the effectiveness and broad applicability of our method compared with several strong baselines.