Rethinking the Necessity of Adaptive Retrieval-Augmented Generation through the Lens of Adaptive Listwise Ranking

TL;DR

This paper reevaluates the need for adaptive retrieval in large language models, proposing AdaRankLLM to improve efficiency and noise filtering, especially for weaker models, through a novel adaptive ranking framework and distillation techniques.

Contribution

It introduces AdaRankLLM, a new adaptive retrieval framework with a zero-shot ranker and a two-stage distillation process, enhancing performance and efficiency of LLMs.

Findings

AdaRankLLM outperforms static retrieval strategies across multiple datasets and models.

Adaptive retrieval acts as a noise filter for weaker models and as an efficiency booster for stronger models.

Significant reduction in context overhead achieved without sacrificing performance.

Abstract

Adaptive Retrieval-Augmented Generation aims to mitigate the interference of extraneous noise by dynamically determining the necessity of retrieving supplementary passages. However, as Large Language Models evolve with increasing robustness to noise, the necessity of adaptive retrieval warrants re-evaluation. In this paper, we rethink this necessity and propose AdaRankLLM, a novel adaptive retrieval framework. To effectively verify the necessity of adaptive listwise reranking, we first develop an adaptive ranker employing a zero-shot prompt with a passage dropout mechanism, and compare its generation outcomes against static fixed-depth retrieval strategies. Furthermore, to endow smaller open-source LLMs with this precise listwise ranking and adaptive filtering capability, we introduce a two-stage progressive distillation paradigm enhanced by data sampling and augmentation techniques. Extensive experiments across three datasets and eight LLMs demonstrate that AdaRankLLM consistently achieves optimal performance in most scenarios with significantly reduced context overhead. Crucially, our analysis reveals a role shift in adaptive retrieval: it functions as a critical noise filter for weaker models to overcome their limitations, while serving as a cost-effective efficiency optimizer for stronger reasoning models.