Do not think about pink elephant!
Kyomin Hwang, Suyoung Kim, JunHoo Lee, Nojun Kwak

TL;DR
This paper reveals that large models like Stable Diffusion and DALL-E3 exhibit the 'white bear phenomenon' similar to human cognition, and proposes prompt-based attacks and defenses to address this vulnerability.
Contribution
It identifies the 'white bear phenomenon' in large models and introduces prompt-based attack and defense methods inspired by cognitive therapy.
Findings
Prompt-based attacks can generate prohibited images.
Defense strategies reduce attack success by up to 48.22%.
Analysis of representation space explains the phenomenon.
Abstract
Large Models (LMs) have heightened expectations for the potential of general AI as they are akin to human intelligence. This paper shows that recent large models such as Stable Diffusion and DALL-E3 also share the vulnerability of human intelligence, namely the "white bear phenomenon". We investigate the causes of the white bear phenomenon by analyzing their representation space. Based on this analysis, we propose a simple prompt-based attack method, which generates figures prohibited by the LM provider's policy. To counter these attacks, we introduce prompt-based defense strategies inspired by cognitive therapy techniques, successfully mitigating attacks by up to 48.22\%.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdversarial Robustness in Machine Learning · Advanced Malware Detection Techniques · User Authentication and Security Systems
MethodsDiffusion
