Towards More Robust Retrieval-Augmented Generation: Evaluating RAG Under Adversarial Poisoning Attacks

TL;DR

This paper empirically investigates the vulnerability of Retrieval-Augmented Generation (RAG) systems to adversarial poisoning attacks and explores methods to improve their robustness and safety in real-world applications.

Contribution

It provides a controlled empirical analysis of RAG under attack, introduces a taxonomy of context types, evaluates retriever vulnerabilities, and proposes skeptical prompting as a partial defense.

Findings

Skeptical prompting activates LLMs' reasoning for self-defense.

Retrievers vary in exposing models to adversarial contexts.

Robustness can be improved with targeted strategies.

Abstract

Retrieval-Augmented Generation (RAG) systems have emerged as a promising solution to mitigate LLM hallucinations and enhance their performance in knowledge-intensive domains. However, these systems are vulnerable to adversarial poisoning attacks, where malicious passages injected into the retrieval corpus can mislead models into producing factually incorrect outputs. In this paper, we present a rigorously controlled empirical study of how RAG systems behave under such attacks and how their robustness can be improved. On the generation side, we introduce a structured taxonomy of context types-adversarial, untouched, and guiding-and systematically analyze their individual and combined effects on model outputs. On the retrieval side, we evaluate several retrievers to measure how easily they expose LLMs to adversarial contexts. Our findings also reveal that "skeptical prompting" can activate LLMs' internal reasoning, enabling partial self-defense against adversarial passages, though its effectiveness depends strongly on the model's reasoning capacity. Together, our experiments (code available at https://github.com/JinyanSu1/eval_PoisonRaG) and analysis provide actionable insights for designing safer and more resilient RAG systems, paving the way for more reliable real-world deployments.