Mitigating Lost-in-Retrieval Problems in Retrieval Augmented Multi-Hop Question Answering

TL;DR

This paper introduces ChainRAG, a progressive retrieval and rewriting method that addresses the lost-in-retrieval problem in retrieval-augmented multi-hop question answering, significantly improving accuracy and efficiency.

Contribution

We propose ChainRAG, a novel sequential retrieval and rewriting approach that mitigates lost-in-retrieval issues in multi-hop QA, enhancing retrieval accuracy and reasoning chain integrity.

Findings

ChainRAG outperforms baselines on MuSiQue, 2Wiki, and HotpotQA datasets.

It improves retrieval effectiveness and answer accuracy across multiple large language models.

The method demonstrates both higher efficiency and robustness in multi-hop QA tasks.

Abstract

In this paper, we identify a critical problem, "lost-in-retrieval", in retrieval-augmented multi-hop question answering (QA): the key entities are missed in LLMs' sub-question decomposition. "Lost-in-retrieval" significantly degrades the retrieval performance, which disrupts the reasoning chain and leads to the incorrect answers. To resolve this problem, we propose a progressive retrieval and rewriting method, namely ChainRAG, which sequentially handles each sub-question by completing missing key entities and retrieving relevant sentences from a sentence graph for answer generation. Each step in our retrieval and rewriting process builds upon the previous one, creating a seamless chain that leads to accurate retrieval and answers. Finally, all retrieved sentences and sub-question answers are integrated to generate a comprehensive answer to the original question. We evaluate ChainRAG on three multi-hop QA datasets - MuSiQue, 2Wiki, and HotpotQA - using three large language models: GPT4o-mini, Qwen2.5-72B, and GLM-4-Plus. Empirical results demonstrate that ChainRAG consistently outperforms baselines in both effectiveness and efficiency.