Understanding and Enhancing the Transferability of Jailbreaking Attacks

TL;DR

This paper investigates the transferability of jailbreaking attacks on large language models, revealing their limitations and proposing a new method, PiF, to improve attack transferability and model evaluation.

Contribution

The paper introduces the PiF method, which enhances attack transferability by dispersing model focus, addressing overfitting issues in adversarial sequences for better LLM vulnerability assessment.

Findings

Adversarial sequences overfit source LLMs, limiting transferability.

PiF effectively disperses focus, improving attack transferability.

PiF enhances red-teaming evaluation of proprietary LLMs.

Abstract

Jailbreaking attacks can effectively manipulate open-source large language models (LLMs) to produce harmful responses. However, these attacks exhibit limited transferability, failing to disrupt proprietary LLMs consistently. To reliably identify vulnerabilities in proprietary LLMs, this work investigates the transferability of jailbreaking attacks by analysing their impact on the model's intent perception. By incorporating adversarial sequences, these attacks can redirect the source LLM's focus away from malicious-intent tokens in the original input, thereby obstructing the model's intent recognition and eliciting harmful responses. Nevertheless, these adversarial sequences fail to mislead the target LLM's intent perception, allowing the target LLM to refocus on malicious-intent tokens and abstain from responding. Our analysis further reveals the inherent distributional dependency within the generated adversarial sequences, whose effectiveness stems from overfitting the source LLM's parameters, resulting in limited transferability to target LLMs. To this end, we propose the Perceived-importance Flatten (PiF) method, which uniformly disperses the model's focus across neutral-intent tokens in the original input, thus obscuring malicious-intent tokens without relying on overfitted adversarial sequences. Extensive experiments demonstrate that PiF provides an effective and efficient red-teaming evaluation for proprietary LLMs.