PruneRAG: Confidence-Guided Query Decomposition Trees for Efficient Retrieval-Augmented Generation

TL;DR

PruneRAG is a novel framework that enhances retrieval-augmented generation by building confidence-guided query trees, improving reasoning stability, evidence utilization, and reducing retrieval costs.

Contribution

It introduces a structured query decomposition approach with adaptive expansion, confidence-based pruning, and entity-level retrieval to address evidence forgetting and inefficiency in RAG systems.

Findings

Achieves higher accuracy on multi-hop QA benchmarks.

Reduces retrieval overhead significantly.

Maintains salient evidence during reasoning processes.

Abstract

Retrieval-augmented generation (RAG) has become a powerful framework for enhancing large language models in knowledge-intensive and reasoning tasks. However, as reasoning chains deepen or search trees expand, RAG systems often face two persistent failures: evidence forgetting, where retrieved knowledge is not effectively used, and inefficiency, caused by uncontrolled query expansions and redundant retrieval. These issues reveal a critical gap between retrieval and evidence utilization in current RAG architectures. We propose PruneRAG, a confidence-guided query decomposition framework that builds a structured query decomposition tree to perform stable and efficient reasoning. PruneRAG introduces three key mechanisms: adaptive node expansion that regulates tree width and depth, confidence-guided decisions that accept reliable answers and prune uncertain branches, and fine-grained retrieval that extracts entity-level anchors to improve retrieval precision. Together, these components preserve salient evidence throughout multi-hop reasoning while significantly reducing retrieval overhead. To better analyze evidence misuse, we define the Evidence Forgetting Rate as a metric to quantify cases where golden evidence is retrieved but not correctly used. Extensive experiments across various multi-hop QA benchmarks show that PruneRAG achieves superior accuracy and efficiency over state-of-the-art baselines.