AdvPrefix: An Objective for Nuanced LLM Jailbreaks

TL;DR

AdvPrefix is a novel prefix-forcing objective that enhances the effectiveness and flexibility of jailbreak attacks on large language models by selecting optimal prefixes based on success rates and likelihood.

Contribution

It introduces a plug-and-play prefix-forcing objective that improves jailbreak success and control, addressing limitations of previous methods.

Findings

Improves nuanced attack success rates from 14% to 80% on Llama-3.

Seamlessly integrates into existing jailbreak attacks.

Reveals current safety alignment limitations.

Abstract

Many jailbreak attacks on large language models (LLMs) rely on a common objective: making the model respond with the prefix ``Sure, here is (harmful request)''. While straightforward, this objective has two limitations: limited control over model behaviors, yielding incomplete or unrealistic jailbroken responses, and a rigid format that hinders optimization. We introduce AdvPrefix, a plug-and-play prefix-forcing objective that selects one or more model-dependent prefixes by combining two criteria: high prefilling attack success rates and low negative log-likelihood. AdvPrefix integrates seamlessly into existing jailbreak attacks to mitigate the previous limitations for free. For example, replacing GCG's default prefixes on Llama-3 improves nuanced attack success rates from 14% to 80%, revealing that current safety alignment fails to generalize to new prefixes. Code and selected prefixes are released at github.com/facebookresearch/jailbreak-objectives.