Adaptive Query Rewriting: Aligning Rewriters through Marginal Probability of Conversational Answers

TL;DR

AdaQR is a novel framework that improves conversational query rewriting by leveraging marginal answer probabilities to align rewriters with retriever preferences, requiring minimal annotations and adapting well across domains.

Contribution

Introduces AdaQR, a training method for query rewriters that uses answer probabilities to optimize without extensive annotations, enhancing adaptability and performance.

Findings

Improves in-domain query rewriting with limited annotations

Effectively adapts to out-of-domain datasets

Enhances passage retrieval in conversational QA

Abstract

Query rewriting is a crucial technique for passage retrieval in open-domain conversational question answering (CQA). It decontexualizes conversational queries into self-contained questions suitable for off-the-shelf retrievers. Existing methods attempt to incorporate retriever's preference during the training of rewriting models. However, these approaches typically rely on extensive annotations such as in-domain rewrites and/or relevant passage labels, limiting the models' generalization and adaptation capabilities. In this paper, we introduce AdaQR ($\textbf{Ada}$ptive $\textbf{Q}$uery $\textbf{R}$ewriting), a framework for training query rewriting models with limited rewrite annotations from seed datasets and completely no passage label. Our approach begins by fine-tuning compact large language models using only ~$10\%$ of rewrite annotations from the seed dataset training split. The models are then utilized to generate rewrite candidates for each query instance. A novel approach is then proposed to assess retriever's preference for these candidates by the probability of answers conditioned on the conversational query by marginalizing the Top-$K$ passages. This serves as the reward for optimizing the rewriter further using Direct Preference Optimization (DPO), a process free of rewrite and retrieval annotations. Experimental results on four open-domain CQA datasets demonstrate that AdaQR not only enhances the in-domain capabilities of the rewriter with limited annotation requirement, but also adapts effectively to out-of-domain datasets.