AdaRewriter: Unleashing the Power of Prompting-based Conversational Query Reformulation via Test-Time Adaptation

TL;DR

AdaRewriter introduces a test-time adaptation framework using a reward model to improve conversational query reformulation, significantly enhancing performance in search tasks, including black-box systems.

Contribution

It presents a novel test-time adaptation method with a lightweight reward model for better query reformulation, outperforming existing approaches.

Findings

Outperforms existing methods on five datasets

Effective in black-box commercial LLM APIs

Demonstrates the potential of test-time adaptation

Abstract

Prompting-based conversational query reformulation has emerged as a powerful approach for conversational search, refining ambiguous user queries into standalone search queries. Best-of-N reformulation over the generated candidates via prompting shows impressive potential scaling capability. However, both the previous tuning methods (training time) and adaptation approaches (test time) can not fully unleash their benefits. In this paper, we propose AdaRewriter, a novel framework for query reformulation using an outcome-supervised reward model via test-time adaptation. By training a lightweight reward model with contrastive ranking loss, AdaRewriter selects the most promising reformulation during inference. Notably, it can operate effectively in black-box systems, including commercial LLM APIs. Experiments on five conversational search datasets show that AdaRewriter significantly outperforms the existing methods across most settings, demonstrating the potential of test-time adaptation for conversational query reformulation.