Against The Achilles' Heel: A Survey on Red Teaming for Generative Models

TL;DR

This survey comprehensively reviews red teaming strategies for generative models, introducing a taxonomy and framework, and exploring emerging topics like multimodal attacks, LLM-based agent risks, and safety-harm trade-offs.

Contribution

It provides a detailed taxonomy of attack strategies, the unified 'searcher' framework, and covers new research areas in generative model safety and robustness.

Findings

Developed a taxonomy of attack strategies based on language model capabilities.

Created the 'searcher' framework to unify automatic red teaming methods.

Explored emerging topics such as multimodal attacks and safety-harm balance.

Abstract

Generative models are rapidly gaining popularity and being integrated into everyday applications, raising concerns over their safe use as various vulnerabilities are exposed. In light of this, the field of red teaming is undergoing fast-paced growth, highlighting the need for a comprehensive survey covering the entire pipeline and addressing emerging topics. Our extensive survey, which examines over 120 papers, introduces a taxonomy of fine-grained attack strategies grounded in the inherent capabilities of language models. Additionally, we have developed the "searcher" framework to unify various automatic red teaming approaches. Moreover, our survey covers novel areas including multimodal attacks and defenses, risks around LLM-based agents, overkill of harmless queries, and the balance between harmlessness and helpfulness.