Prompts have evil twins
Rimon Melamed, Lucas H. McCabe, Tanay Wakhare, Yejin Kim, H. Howie, Huang, Enric Boix-Adsera

TL;DR
This paper introduces 'evil twins', obfuscated prompts that are unintelligible to humans but elicit similar responses in language models, revealing vulnerabilities and transferability across models.
Contribution
It presents a method to generate 'evil twin' prompts that mimic natural prompts' behavior while being uninterpretable, highlighting new security concerns.
Findings
Evil twins can be generated for various prompts.
They transfer between different language models.
Obfuscated prompts can replicate original behavior.
Abstract
We discover that many natural-language prompts can be replaced by corresponding prompts that are unintelligible to humans but that provably elicit similar behavior in language models. We call these prompts "evil twins" because they are obfuscated and uninterpretable (evil), but at the same time mimic the functionality of the original natural-language prompts (twins). Remarkably, evil twins transfer between models. We find these prompts by solving a maximum-likelihood problem which has applications of independent interest.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
Taxonomy
TopicsTopic Modeling · Natural Language Processing Techniques · Speech and dialogue systems
