SentinelAgent: Graph-based Anomaly Detection in Multi-Agent Systems

TL;DR

SentinelAgent introduces a graph-based, system-level anomaly detection framework for multi-agent systems, leveraging runtime behavioral analysis and LLM-powered oversight to identify complex security threats and systemic failures.

Contribution

This work presents a novel graph-based modeling approach combined with an LLM-powered oversight agent for comprehensive anomaly detection in multi-agent systems.

Findings

Effective detection of prompt injections and single-point faults.

Identification of multi-agent collusion and latent exploit paths.

Demonstrated success in real-world case studies.

Abstract

The rise of large language model (LLM)-based multi-agent systems (MAS) introduces new security and reliability challenges. While these systems show great promise in decomposing and coordinating complex tasks, they also face multi-faceted risks across prompt manipulation, unsafe tool usage, and emergent agent miscoordination. Existing guardrail mechanisms offer only partial protection, primarily at the input-output level, and fall short in addressing systemic or multi-point failures in MAS. In this work, we present a system-level anomaly detection framework tailored for MAS, integrating structural modeling with runtime behavioral oversight. Our approach consists of two components. First, we propose a graph-based framework that models agent interactions as dynamic execution graphs, enabling semantic anomaly detection at node, edge, and path levels. Second, we introduce a pluggable SentinelAgent, an LLM-powered oversight agent that observes, analyzes, and intervenes in MAS execution based on security policies and contextual reasoning. By bridging abstract detection logic with actionable enforcement, our method detects not only single-point faults and prompt injections but also multi-agent collusion and latent exploit paths. We validate our framework through two case studies, including an email assistant and Microsoft's Magentic-One system, demonstrating its ability to detect covert risks and provide explainable root-cause attribution. Our work lays the foundation for more trustworthy, monitorable, and secure agent-based AI ecosystems.