ProGRank: Probe-Gradient Reranking to Defend Dense-Retriever RAG from Corpus Poisoning

TL;DR

ProGRank is a training-free, retriever-side defense method for dense-retriever RAG systems that detects and reranks potentially poisoned passages using probe-gradient signals, enhancing robustness without retraining.

Contribution

ProGRank introduces a novel, post hoc, training-free reranking approach that improves defense against corpus poisoning in RAG systems by leveraging probe-gradient derived instability signals.

Findings

ProGRank outperforms existing defenses across multiple datasets and attack types.

It maintains high utility while significantly improving robustness against corpus poisoning.

ProGRank remains effective under adaptive evasive attacks.

Abstract

Retrieval-Augmented Generation (RAG) improves the reliability of large language model applications by grounding generation in retrieved evidence, but it also introduces a new attack surface: corpus poisoning. In this setting, an adversary injects or edits passages so that they are ranked into the Top-$K$ results for target queries and then affect downstream generation. Existing defences against corpus poisoning often rely on content filtering, auxiliary models, or generator-side reasoning, which can make deployment more difficult. We propose ProGRank, a post hoc, training-free retriever-side defence for dense-retriever RAG. ProGRank stress-tests each query--passage pair under mild randomized perturbations and extracts probe gradients from a small fixed parameter subset of the retriever. From these signals, it derives two instability signals, representational consistency and dispersion risk, and combines them with a score gate in a reranking step. ProGRank preserves the original passage content, requires no retraining, and also supports a surrogate-based variant when the deployed retriever is unavailable. Extensive experiments across three datasets, three dense retriever backbones, representative corpus poisoning attacks, and both retrieval-stage and end-to-end settings show that ProGRank provides stronger defence performance and a favorable robustness--utility trade-off. It also remains competitive under adaptive evasive attacks.