Agent Security is a Systems Problem

TL;DR

This paper argues that agent security should be addressed as a systems problem, emphasizing system-level security invariants and principles to prevent attacks on AI agents.

Contribution

It introduces a systems security perspective for agent security, integrating principles from cybersecurity to enhance robustness beyond model improvements.

Findings

Analyzed eleven real-world agent attacks and identified prevention strategies.

Highlighted the importance of system-level security invariants.

Provided foundational principles for designing secure agent systems.

Abstract

We take the position that agent security must be approached as a systems problem: the AI model powering the agent must be treated as an untrusted component, and security invariants must be enforced at the system level. Through this lens, efforts to increase model robustness (the dominant viewpoint in the community) are insufficient on their own. Instead, we must complement existing efforts with techniques from the systems security domain. Based on our experience as cybersecurity researchers in operating systems, networks, formal methods, and adversarial machine learning, we articulate a set of core principles, grounded in decades of systems security research, that provide a foundation for designing agentic systems with predictable guarantees. As evidence, we analyze eleven representative real-world attacks on agents and discuss how systems principles, if realized, could have prevented these attacks. We also identify the research challenges that stand in the way of implementing these principles in agents.