Response Attack: Exploiting Contextual Priming to Jailbreak Large Language Models

TL;DR

This paper reveals a new vulnerability in large language models where previous dialogue responses can be exploited to induce policy-violating outputs, introducing a novel attack method called Response Attack that outperforms existing jailbreak techniques.

Contribution

The paper introduces Response Attack, a novel framework that leverages intermediate responses to effectively jailbreak LLMs, demonstrating higher success rates than existing methods.

Findings

Response Attack achieves higher success rates than baseline methods.

Intermediate responses significantly influence model outputs.

The attack maintains stealth and efficiency across multiple LLMs.

Abstract

Contextual priming, where earlier stimuli covertly bias later judgments, offers an unexplored attack surface for large language models (LLMs). We uncover a contextual priming vulnerability in which the previous response in the dialogue can steer its subsequent behavior toward policy-violating content. While existing jailbreak attacks largely rely on single-turn or multi-turn prompt manipulations, or inject static in-context examples, these methods suffer from limited effectiveness, inefficiency, or semantic drift. We introduce Response Attack (RA), a novel framework that strategically leverages intermediate, mildly harmful responses as contextual primers within a dialogue. By reformulating harmful queries and injecting these intermediate responses before issuing a targeted trigger prompt, RA exploits a previously overlooked vulnerability in LLMs. Extensive experiments across eight state-of-the-art LLMs show that RA consistently achieves significantly higher attack success rates than nine leading jailbreak baselines. Our results demonstrate that the success of RA is directly attributable to the strategic use of intermediate responses, which induce models to generate more explicit and relevant harmful content while maintaining stealth, efficiency, and fidelity to the original query. The code and data are available at https://github.com/Dtc7w3PQ/Response-Attack.