Quality-Diversity Red-Teaming: Automated Generation of High-Quality and Diverse Attackers for Large Language Models

TL;DR

This paper introduces QDRT, a novel framework for automated red-teaming of large language models that produces diverse, high-quality adversarial prompts by training multiple specialized attackers with goal-driven behavior.

Contribution

QDRT addresses limitations of previous methods by using behavior-conditioned training and multiple attackers, enhancing diversity and effectiveness in safety evaluations of LLMs.

Findings

QDRT generates more diverse adversarial prompts than prior methods.

QDRT's attacks are more effective across various LLMs like GPT-2, Llama-3, Gemma-2, and Qwen2.5.

The framework improves safety assessment coverage for large language models.

Abstract

Ensuring safety of large language models (LLMs) is important. Red teaming--a systematic approach to identifying adversarial prompts that elicit harmful responses from target LLMs--has emerged as a crucial safety evaluation method. Within this framework, the diversity of adversarial prompts is essential for comprehensive safety assessments. We find that previous approaches to red-teaming may suffer from two key limitations. First, they often pursue diversity through simplistic metrics like word frequency or sentence embedding similarity, which may not capture meaningful variation in attack strategies. Second, the common practice of training a single attacker model restricts coverage across potential attack styles and risk categories. This paper introduces Quality-Diversity Red-Teaming (QDRT), a new framework designed to address these limitations. QDRT achieves goal-driven diversity through behavior-conditioned training and implements a behavioral replay buffer in an open-ended manner. Additionally, it trains multiple specialized attackers capable of generating high-quality attacks across diverse styles and risk categories. Our empirical evaluation demonstrates that QDRT generates attacks that are both more diverse and more effective against a wide range of target LLMs, including GPT-2, Llama-3, Gemma-2, and Qwen2.5. This work advances the field of LLM safety by providing a systematic and effective approach to automated red-teaming, ultimately supporting the responsible deployment of LLMs.