SafeAgent: A Runtime Protection Architecture for Agentic Systems

TL;DR

SafeAgent is a runtime security architecture for LLM agents that enhances robustness against prompt-injection attacks by managing interaction trajectories and separating governance from risk reasoning.

Contribution

It introduces a novel architecture with a runtime controller and context-aware decision core to improve agent safety and robustness.

Findings

SafeAgent outperforms baseline and guardrail methods in robustness.

Experiments on ASB and InjecAgent demonstrate improved security.

Recovery confidence and policy weighting influence safety-utility trade-offs.

Abstract

Large language model (LLM) agents are vulnerable to prompt-injection attacks that propagate through multi-step workflows, tool interactions, and persistent context, making input-output filtering alone insufficient for reliable protection. This paper presents SafeAgent, a runtime security architecture that treats agent safety as a stateful decision problem over evolving interaction trajectories. The proposed design separates execution governance from semantic risk reasoning through two coordinated components: a runtime controller that mediates actions around the agent loop and a context-aware decision core that operates over persistent session state. The core is formalized as a context-aware advanced machine intelligence and instantiated through operators for risk encoding, utility-cost evaluation, consequence modeling, policy arbitration, and state synchronization. Experiments on Agent Security Bench (ASB) and InjecAgent show that SafeAgent consistently improves robustness over baseline and text-level guardrail methods while maintaining competitive benign-task performance. Ablation studies further show that recovery confidence and policy weighting determine distinct safety-utility operating points.