
TL;DR
This paper introduces the Attention Redistribution Attack (ARA), a white-box adversarial method that manipulates attention mechanisms in large language models to bypass safety measures with minimal tokens.
Contribution
The paper presents ARA, a novel attack targeting attention geometry to bypass safety alignment, revealing safety emerges from attention routing rather than isolated heads.
Findings
ARA bypasses safety with as few as 5 tokens and 500 steps
Achieves 36% attack success rate on Mistral-7B and 30% on LLaMA-3
Safety heads are not isolated; attention rerouting causes safety failures
Abstract
Safety-aligned large language models rely on RLHF and instruction tuning to refuse harmful requests, yet the internal mechanisms implementing safety behavior remain poorly understood. We introduce the Attention Redistribution Attack (ARA), a white-box adversarial attack that identifies safety-critical attention heads and crafts nonsemantic adversarial tokens that redirect attention away from safety-relevant positions. Unlike prior jailbreak methods operating at the semantic or output-logit level, ARA targets the geometry of softmax attention on the probability simplex using Gumbel-softmax optimization over targeted heads. Across LLaMA-3-8B-Instruct, Mistral-7B-Instruct-v0.1, and Gemma-2-9B-it, ARA bypasses safety alignment with as few as 5 tokens and 500 optimization steps, achieving 36% ASR on Mistral-7B and 30% on LLaMA-3 against 200 HarmBench prompts, while Gemma-2 remains at 1%. Our…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
