RAG LLMs are Not Safer: A Safety Analysis of Retrieval-Augmented Generation for Large Language Models

TL;DR

This paper investigates how Retrieval-Augmented Generation (RAG) frameworks impact the safety of large language models, revealing that RAG can reduce safety and alter safety profiles, necessitating specialized safety measures.

Contribution

It provides the first detailed comparison of RAG and non-RAG LLMs regarding safety, identifying safety risks and evaluating red-teaming effectiveness specific to RAG settings.

Findings

RAG can decrease model safety and alter safety profiles.

Safe models with safe documents can still produce unsafe outputs.

Existing red-teaming methods are less effective for RAG models.

Abstract

Efforts to ensure the safety of large language models (LLMs) include safety fine-tuning, evaluation, and red teaming. However, despite the widespread use of the Retrieval-Augmented Generation (RAG) framework, AI safety work focuses on standard LLMs, which means we know little about how RAG use cases change a model's safety profile. We conduct a detailed comparative analysis of RAG and non-RAG frameworks with eleven LLMs. We find that RAG can make models less safe and change their safety profile. We explore the causes of this change and find that even combinations of safe models with safe documents can cause unsafe generations. In addition, we evaluate some existing red teaming methods for RAG settings and show that they are less effective than when used for non-RAG settings. Our work highlights the need for safety research and red-teaming methods specifically tailored for RAG LLMs.