Sherlock Your Queries: Learning to Ask the Right Questions for Dialogue-Based Retrieval

TL;DR

SherlockLLM is a reinforcement learning-based dialogue system that learns to ask the most informative questions to improve retrieval accuracy without needing large annotated datasets.

Contribution

The paper introduces SherlockLLM, a novel RL framework that learns optimal questioning strategies for dialogue-based retrieval, reducing reliance on annotated dialogue data.

Findings

Matches strong baselines on structured tasks

Approaches theoretical optimal in binary search

Outperforms baselines on unstructured tasks

Abstract

User queries in information retrieval are often ambiguous, making it challenging for systems to identify a user's target from a single query. While recent dialogue-based interactive retrieval systems can clarify user intent, they are inefficient as they often lack an explicit strategy to ask the most informative questions. To address this limitation, we propose SherlockLLM, a dialogue-driven retrieval framework that learns an optimal questioning strategy via Reinforcement Learning (RL) and avoids the need for large-scale annotated dialogue data. In our framework, an agent is trained to generate a sequence of binary questions to efficiently narrow down the search space. To validate our approach, we introduce a benchmark with both structured and unstructured tasks. Experimental results show that SherlockLLM is a robust and efficient solution. On the structured tasks, its performance matches strong baselines and approaches the theoretical optimal defined by binary search. On the challenging unstructured task, our agent significantly outperforms these baselines, showcasing its ability to learn a highly effective information-seeking dialogue policy.