On the Vulnerability of Applying Retrieval-Augmented Generation within Knowledge-Intensive Application Domains

TL;DR

This paper reveals that retrieval-augmented generation systems are vulnerable to universal poisoning attacks in knowledge-intensive domains, and proposes an effective detection-based defense to mitigate this risk.

Contribution

The study uncovers a novel vulnerability in RAG systems related to poisoned document retrieval and introduces a new detection method to enhance their robustness.

Findings

Retrieval systems are susceptible to universal poisoning attacks in medical Q&A.

Poisoned documents can be accurately retrieved using attacker-specified queries.

The proposed detection method achieves high detection rates across various domains.

Abstract

Retrieval-Augmented Generation (RAG) has been empirically shown to enhance the performance of large language models (LLMs) in knowledge-intensive domains such as healthcare, finance, and legal contexts. Given a query, RAG retrieves relevant documents from a corpus and integrates them into the LLMs' generation process. In this study, we investigate the adversarial robustness of RAG, focusing specifically on examining the retrieval system. First, across 225 different setup combinations of corpus, retriever, query, and targeted information, we show that retrieval systems are vulnerable to universal poisoning attacks in medical Q\&A. In such attacks, adversaries generate poisoned documents containing a broad spectrum of targeted information, such as personally identifiable information. When these poisoned documents are inserted into a corpus, they can be accurately retrieved by any users, as long as attacker-specified queries are used. To understand this vulnerability, we discovered that the deviation from the query's embedding to that of the poisoned document tends to follow a pattern in which the high similarity between the poisoned document and the query is retained, thereby enabling precise retrieval. Based on these findings, we develop a new detection-based defense to ensure the safe use of RAG. Through extensive experiments spanning various Q\&A domains, we observed that our proposed method consistently achieves excellent detection rates in nearly all cases.