From Seed to Harvest: Augmenting Human Creativity with AI for Red-teaming Text-to-Image Models

TL;DR

This paper introduces Seed2Harvest, a hybrid approach combining human and AI-generated prompts to improve the diversity and effectiveness of adversarial testing for text-to-image models, enhancing safety evaluation.

Contribution

We propose Seed2Harvest, a novel hybrid method that expands human-crafted adversarial prompts with AI, increasing diversity and maintaining attack success rates for robust T2I model evaluation.

Findings

Expanded dataset has 535 unique locations, significantly more than original.

Achieved higher Shannon entropy, indicating greater diversity.

Maintained comparable attack success rates to original prompts.

Abstract

Text-to-image (T2I) models have become prevalent across numerous applications, making their robust evaluation against adversarial attacks a critical priority. Continuous access to new and challenging adversarial prompts across diverse domains is essential for stress-testing these models for resilience against novel attacks from multiple vectors. Current techniques for generating such prompts are either entirely authored by humans or synthetically generated. On the one hand, datasets of human-crafted adversarial prompts are often too small in size and imbalanced in their cultural and contextual representation. On the other hand, datasets of synthetically-generated prompts achieve scale, but typically lack the realistic nuances and creative adversarial strategies found in human-crafted prompts. To combine the strengths of both human and machine approaches, we propose Seed2Harvest, a hybrid red-teaming method for guided expansion of culturally diverse, human-crafted adversarial prompt seeds. The resulting prompts preserve the characteristics and attack patterns of human prompts while maintaining comparable average attack success rates (0.31 NudeNet, 0.36 SD NSFW, 0.12 Q16). Our expanded dataset achieves substantially higher diversity with 535 unique geographic locations and a Shannon entropy of 7.48, compared to 58 locations and 5.28 entropy in the original dataset. Our work demonstrates the importance of human-machine collaboration in leveraging human creativity and machine computational capacity to achieve comprehensive, scalable red-teaming for continuous T2I model safety evaluation.