Scaling DPPs for RAG: Density Meets Diversity

TL;DR

This paper introduces ScalDPP, a scalable DPP-based retrieval method for RAG that jointly optimizes for density and diversity, improving evidence relevance and coverage.

Contribution

It proposes a novel DPP-based retrieval mechanism with a lightweight adapter and a new set-level loss to enhance RAG performance.

Findings

ScalDPP outperforms standard relevance ranking in experiments.

The method effectively balances density and diversity in retrieved evidence.

Experimental results validate the superiority of ScalDPP.

Abstract

Retrieval-Augmented Generation (RAG) enhances Large Language Models (LLMs) by grounding generation in external knowledge, yielding relevance responses that are aligned with factual evidence and evolving corpora. Standard RAG pipelines construct context through relevance ranking, performing point-wise scoring between the user query and each corpora chunk. This formulation, however, ignores interactions among retrieved candidates, leading to redundant contexts that dilute density and fail to surface complementary evidence. We argue that effective retrieval should optimize jointly for both density and diversity, ensuring the grounding evidence that is dense in information yet diverse in coverage. In this study, we propose ScalDPP, a diversity-aware retrieval mechanism for RAG that incorporates Determinantal Point Processes (DPPs) through a lightweight P-Adapter, enabling scalable modeling of inter-chunk dependencies and complementary context selection. In addition, we develop a novel set-level objective, Diverse Margin Loss (DML), that enforces ground-truth complementary evidence chains to dominate any equally sized redundant alternatives under DPP geometry. Experimental results demonstrate the superiority of ScalDPP, substantiating our core statement in practice.