Stop Fixating on Prompts: Reasoning Hijacking and Constraint Tightening for Red-Teaming LLM Agents

TL;DR

This paper introduces JailAgent, a novel framework for red-teaming LLM agents by implicitly manipulating their reasoning and memory without altering user prompts, enhancing adaptability and security.

Contribution

JailAgent is the first approach to avoid prompt modification by implicitly hijacking reasoning and tightening constraints for robust red-teaming of LLM agents.

Findings

JailAgent outperforms existing methods across multiple models and scenarios.

It effectively manipulates reasoning trajectories and memory retrieval.

The framework demonstrates high adaptability and security in diverse environments.

Abstract

With the widespread application of LLM-based agents across various domains, their complexity has introduced new security threats. Existing red-team methods mostly rely on modifying user prompts, which lack adaptability to new data and may impact the agent's performance. To address the challenge, this paper proposes the JailAgent framework, which completely avoids modifying the user prompt. Specifically, it implicitly manipulates the agent's reasoning trajectory and memory retrieval with three key stages: Trigger Extraction, Reasoning Hijacking, and Constraint Tightening. Through precise trigger identification, real-time adaptive mechanisms, and an optimized objective function, JailAgent demonstrates outstanding performance in cross-model and cross-scenario environments.