Zombie Agents: Persistent Control of Self-Evolving LLM Agents via Self-Reinforcing Injections

TL;DR

This paper investigates security vulnerabilities in self-evolving LLM agents, demonstrating how malicious payloads can persist across sessions and cause unauthorized actions, highlighting the need for improved defenses.

Contribution

It formalizes the Zombie Agent attack, introduces a black-box attack framework, and proposes persistence strategies to defend against long-term memory injections in LLM agents.

Findings

Memory injection can persist over multiple sessions.

Current defenses are insufficient against persistent attacks.

Proposed strategies improve resilience of memory systems.

Abstract

Self-evolving LLM agents update their internal state across sessions, often by writing and reusing long-term memory. This design improves performance on long-horizon tasks but creates a security risk: untrusted external content observed during a benign session can be stored as memory and later treated as instruction. We study this risk and formalize a persistent attack we call a Zombie Agent, where an attacker covertly implants a payload that survives across sessions, effectively turning the agent into a puppet of the attacker. We present a black-box attack framework that uses only indirect exposure through attacker-controlled web content. The attack has two phases. During infection, the agent reads a poisoned source while completing a benign task and writes the payload into long-term memory through its normal update process. During trigger, the payload is retrieved or carried forward and causes unauthorized tool behavior. We design mechanism-specific persistence strategies for common memory implementations, including sliding-window and retrieval-augmented memory, to resist truncation and relevance filtering. We evaluate the attack on representative agent setups and tasks, measuring both persistence over time and the ability to induce unauthorized actions while preserving benign task quality. Our results show that memory evolution can convert one-time indirect injection into persistent compromise, which suggests that defenses focused only on per-session prompt filtering are not sufficient for self-evolving agents.