BiRD: A Bidirectional Ranking Defense Mechanism for Retrieval Augmented Generation

TL;DR

BiRD is a novel defense mechanism for Retrieval-Augmented Generation that uses bidirectional ranking patterns to improve robustness against adversarial attacks while maintaining efficiency.

Contribution

It introduces a dual-signal framework leveraging forward and backward rankings to detect poisoned documents, addressing limitations of prior semantic-only approaches.

Findings

BiRD reduces PoisonedRAG attack success rate by up to 54%.

BiRD improves task accuracy by up to 56%.

BiRD maintains average latency under 1 second.

Abstract

The growing adoption of Retrieval-Augmented Generation (RAG) has led to a rise in adversarial attacks. Existing defenses, relying on semantic analysis or voting, face a trade-off between high computational cost and limited robustness under strong poisoning attacks. Their fundamental limitation is the exclusive focus on semantic content relevance, while neglecting the retrieval context that is critically defined by ranking structures. To this end, we investigate the bidirectional ranking behavior of poisoned and benign documents, and discover a key discriminative pattern: poisoned documents exhibit significantly stronger alignment between their backward rankings and the query's forward ranking. Capitalizing on this, we propose BiRD, a bidirectional ranking defense mechanism built upon a dual-signal framework that leverages forward ranking to assess semantic content relevance and backward ranking to quantify ranking context consistency. This design directly addresses the fundamental limitation of prior approaches, enabling simultaneous efficiency and robustness. Extensive evaluation across 3 datasets with 3 retrievers and 3 LLMs under 2 attack scenarios validates BiRD's effectiveness. Notably, BiRD reduces the attack success rate of PoisonedRAG by up to 54% while simultaneously improving task accuracy by up to 56%, with average additional latency under 1 second.