FLIRT: Feedback Loop In-context Red Teaming

TL;DR

This paper introduces FLIRT, an automatic feedback loop-based red teaming framework that tests the robustness of generative models, including text-to-image and text-to-text models, by exposing vulnerabilities to unsafe content generation.

Contribution

We propose a novel in-context learning feedback loop for automatic red teaming, effectively uncovering vulnerabilities in black-box generative models against unsafe outputs.

Findings

Stable Diffusion models are vulnerable despite safety features

The framework effectively red teams text-to-text models

Adversarial prompts can trigger unsafe content generation

Abstract

Warning: this paper contains content that may be inappropriate or offensive. As generative models become available for public use in various applications, testing and analyzing vulnerabilities of these models has become a priority. In this work, we propose an automatic red teaming framework that evaluates a given black-box model and exposes its vulnerabilities against unsafe and inappropriate content generation. Our framework uses in-context learning in a feedback loop to red team models and trigger them into unsafe content generation. In particular, taking text-to-image models as target models, we explore different feedback mechanisms to automatically learn effective and diverse adversarial prompts. Our experiments demonstrate that even with enhanced safety features, Stable Diffusion (SD) models are vulnerable to our adversarial prompts, raising concerns on their robustness in practical uses. Furthermore, we demonstrate that the proposed framework is effective for red teaming text-to-text models.