Stable Diffusion is Unstable

TL;DR

This paper reveals the fragility of text-to-image models to small prompt perturbations and introduces ATM, a gradient-based attack method that effectively exploits this vulnerability, achieving high success rates in disrupting image generation.

Contribution

The paper presents ATM, a novel gradient-based attack method that learns a Gumbel Softmax distribution for efficient and effective prompt perturbations in text-to-image models.

Findings

ATM achieves 91.1% success in short-text attacks.

ATM achieves 81.2% success in long-text attacks.

Identifies four attack patterns based on generation speed, characteristics, polysemy, and word positioning.

Abstract

Recently, text-to-image models have been thriving. Despite their powerful generative capacity, our research has uncovered a lack of robustness in this generation process. Specifically, the introduction of small perturbations to the text prompts can result in the blending of primary subjects with other categories or their complete disappearance in the generated images. In this paper, we propose Auto-attack on Text-to-image Models (ATM), a gradient-based approach, to effectively and efficiently generate such perturbations. By learning a Gumbel Softmax distribution, we can make the discrete process of word replacement or extension continuous, thus ensuring the differentiability of the perturbation generation. Once the distribution is learned, ATM can sample multiple attack samples simultaneously. These attack samples can prevent the generative model from generating the desired subjects without compromising image quality. ATM has achieved a 91.1% success rate in short-text attacks and an 81.2% success rate in long-text attacks. Further empirical analysis revealed four attack patterns based on: 1) the variability in generation speed, 2) the similarity of coarse-grained characteristics, 3) the polysemy of words, and 4) the positioning of words.