DiffusionAttacker: Diffusion-Driven Prompt Manipulation for LLM Jailbreak

TL;DR

This paper presents DiffusionAttacker, a novel diffusion-based method for generating effective LLM jailbreak prompts that outperform existing techniques in success rate, fluency, and diversity.

Contribution

The paper introduces a diffusion-driven, seq2seq approach for prompt rewriting, offering more flexible modifications and improved attack performance over autoregressive methods.

Findings

Outperforms previous methods in attack success rate

Achieves higher fluency and diversity in generated prompts

Demonstrates effectiveness on Advbench and Harmbench datasets

Abstract

Large Language Models (LLMs) are susceptible to generating harmful content when prompted with carefully crafted inputs, a vulnerability known as LLM jailbreaking. As LLMs become more powerful, studying jailbreak methods is critical to enhancing security and aligning models with human values. Traditionally, jailbreak techniques have relied on suffix addition or prompt templates, but these methods suffer from limited attack diversity. This paper introduces DiffusionAttacker, an end-to-end generative approach for jailbreak rewriting inspired by diffusion models. Our method employs a sequence-to-sequence (seq2seq) text diffusion model as a generator, conditioning on the original prompt and guiding the denoising process with a novel attack loss. Unlike previous approaches that use autoregressive LLMs to generate jailbreak prompts, which limit the modification of already generated tokens and restrict the rewriting space, DiffusionAttacker utilizes a seq2seq diffusion model, allowing more flexible token modifications. This approach preserves the semantic content of the original prompt while producing harmful content. Additionally, we leverage the Gumbel-Softmax technique to make the sampling process from the diffusion model's output distribution differentiable, eliminating the need for iterative token search. Extensive experiments on Advbench and Harmbench demonstrate that DiffusionAttacker outperforms previous methods across various evaluation metrics, including attack success rate (ASR), fluency, and diversity.