Shifting from Ranking to Set Selection for Retrieval Augmented Generation

TL;DR

This paper introduces SETR, a set-wise passage selection method for Retrieval-Augmented Generation that improves the collective relevance of retrieved passages for complex, multi-hop questions, outperforming traditional reranking approaches.

Contribution

SETR explicitly models query information needs with Chain-of-Thought reasoning to select an optimal set of passages, advancing retrieval strategies in RAG systems.

Findings

SETR outperforms existing rerankers in answer correctness.

SETR improves retrieval quality on multi-hop RAG benchmarks.

The approach offers an effective alternative to traditional reranking methods.

Abstract

Retrieval in Retrieval-Augmented Generation(RAG) must ensure that retrieved passages are not only individually relevant but also collectively form a comprehensive set. Existing approaches primarily rerank top-k passages based on their individual relevance, often failing to meet the information needs of complex queries in multi-hop question answering. In this work, we propose a set-wise passage selection approach and introduce SETR, which explicitly identifies the information requirements of a query through Chain-of-Thought reasoning and selects an optimal set of passages that collectively satisfy those requirements. Experiments on multi-hop RAG benchmarks show that SETR outperforms both proprietary LLM-based rerankers and open-source baselines in terms of answer correctness and retrieval quality, providing an effective and efficient alternative to traditional rerankers in RAG systems. The code is available at https://github.com/LGAI-Research/SetR