Beyond Suffixes: Token Position in GCG Adversarial Attacks on Large Language Models

TL;DR

This paper investigates how the position of adversarial tokens within prompts affects the success of GCG attacks on LLMs, revealing a critical blind spot in current safety evaluations.

Contribution

It uncovers the significance of token placement in adversarial prompts and demonstrates its impact on attack success, highlighting a gap in robustness assessments.

Findings

Optimizing attacks with prefix tokens increases success rates.

Varying token position during evaluation significantly affects attack outcomes.

Current safety evaluations overlook the importance of token placement.

Abstract

Large Language Models (LLMs) have seen widespread adoption across multiple domains, creating an urgent need for robust safety alignment mechanisms. However, robustness remains challenging due to jailbreak attacks that bypass alignment via adversarial prompts. In this work, we focus on the prevalent Greedy Coordinate Gradient (GCG) attack and identify a previously underexplored attack axis in jailbreak attacks typically framed as suffix-based: the placement of adversarial tokens within the prompt. Using GCG as a case study, we show that both optimizing attacks to generate prefixes instead of suffixes and varying adversarial token position during evaluation substantially influence attack success rates. Our findings highlight a critical blind spot in current safety evaluations and underline the need to account for the position of adversarial tokens in the adversarial robustness evaluation of LLMs.