Retrieval-Augmented Generation by Evidence Retroactivity in LLMs

TL;DR

RetroRAG introduces a retroactive reasoning framework for retrieval-augmented generation, enabling iterative evidence refinement and significantly improving answer accuracy in complex question answering tasks.

Contribution

It pioneers a retroactive reasoning paradigm that revises evidence and updates reasoning chains, advancing beyond traditional unidirectional retrieval methods.

Findings

RetroRAG outperforms existing methods in empirical evaluations.

The framework effectively refines evidence iteratively.

RetroRAG enhances reasoning accuracy in complex questions.

Abstract

Retrieval-augmented generation has gained significant attention due to its ability to integrate relevant external knowledge, enhancing the accuracy and reliability of the LLMs' responses. Most of the existing methods apply a dynamic multiple retrieval-generating process, to address multi-hop complex questions by decomposing them into sub-problems. However, these methods rely on an unidirectional forward reasoning paradigm, where errors from insufficient reasoning steps or inherent flaws in current retrieval systems are irreversible, potentially derailing the entire reasoning chain. For the first time, this work introduces Retroactive Retrieval-Augmented Generation (RetroRAG), a novel framework to build a retroactive reasoning paradigm. RetroRAG revises and updates the evidence, redirecting the reasoning chain to the correct direction. RetroRAG constructs an evidence-collation-discovery framework to search, generate, and refine credible evidence. It synthesizes inferential evidence related to the key entities in the question from the existing source knowledge and formulates search queries to uncover additional information. As new evidence is found, RetroRAG continually updates and organizes this information, enhancing its ability to locate further necessary evidence. Paired with an Answerer to generate and evaluate outputs, RetroRAG is capable of refining its reasoning process iteratively until a reliable answer is obtained. Empirical evaluations show that RetroRAG significantly outperforms existing methods.