Stateful Evidence-Driven Retrieval-Augmented Generation with Iterative Reasoning

TL;DR

This paper introduces a novel retrieval-augmented generation framework that models question answering as a progressive evidence accumulation process, enhancing stability and robustness through iterative reasoning and structured evidence management.

Contribution

It proposes a stateful, evidence-driven RAG framework with iterative query refinement, improving evidence aggregation and robustness over existing methods.

Findings

Achieves consistent improvements on multiple question answering benchmarks.

Effectively accumulates high-quality evidence despite noisy retrieval.

Enhances stability and robustness of RAG models through iterative reasoning.

Abstract

Retrieval-Augmented Generation (RAG) grounds Large Language Models (LLMs) in external knowledge but often suffers from flat context representations and stateless retrieval, leading to unstable performance. We propose Stateful Evidence-Driven RAG with Iterative Reasoning, a framework that models question answering as a progressive evidence accumulation process. Retrieved documents are converted into structured reasoning units with explicit relevance and confidence signals and maintained in a persistent evidence pool capturing both supportive and non-supportive information. The framework performs evidence-driven deficiency analysis to identify gaps and conflicts and iteratively refines queries to guide subsequent retrieval. This iterative reasoning process enables stable evidence aggregation and improves robustness to noisy retrieval. Experiments on multiple question answering benchmarks demonstrate consistent improvements over standard RAG and multi-step baselines, while effectively accumulating high-quality evidence and maintaining stable performance under substantial retrieval noise.