Query Decomposition for RAG: Balancing Exploration-Exploitation

TL;DR

This paper introduces a method for query decomposition in retrieval-augmented generation that balances exploration and exploitation, improving document relevance estimation and long-form generation quality.

Contribution

It formulates query decomposition as a bandit problem, applying bandit learning methods to optimize sub-query selection in RAG systems.

Findings

35% gain in document-level precision

15% increase in α-nDCG

Improved long-form generation performance

Abstract

Retrieval-augmented generation (RAG) systems address complex user requests by decomposing them into subqueries, retrieving potentially relevant documents for each, and then aggregating them to generate an answer. Efficiently selecting informative documents requires balancing a key trade-off: (i) retrieving broadly enough to capture all the relevant material, and (ii) limiting retrieval to avoid excessive noise and computational cost. We formulate query decomposition and document retrieval in an exploitation-exploration setting, where retrieving one document at a time builds a belief about the utility of a given sub-query and informs the decision to continue exploiting or exploring an alternative. We experiment with a variety of bandit learning methods and demonstrate their effectiveness in dynamically selecting the most informative sub-queries. Our main finding is that estimating document relevance using rank information and human judgments yields a 35% gain in document-level precision, 15% increase in {\alpha}-nDCG, and better performance on the downstream task of long-form generation.