ATAG: AI-Agent Application Threat Assessment with Attack Graphs

TL;DR

The paper introduces ATAG, a framework that extends attack graph methods to analyze security risks in AI-agent applications powered by LLMs, addressing the complexity and evolving vulnerabilities of such systems.

Contribution

ATAG extends existing attack graph tools with custom logic to model AI-agent topologies and vulnerabilities, and introduces the LLM Vulnerability Database for standardization.

Findings

Successfully modeled multi-step attack scenarios exploiting LLM vulnerabilities

Demonstrated ATAG's ability to identify complex attack paths in multi-agent systems

Facilitated proactive threat mitigation in AI-agent applications

Abstract

Evaluating the security of multi-agent systems (MASs) powered by large language models (LLMs) is challenging, primarily because of the systems' complex internal dynamics and the evolving nature of LLM vulnerabilities. Traditional attack graph (AG) methods often lack the specific capabilities to model attacks on LLMs. This paper introduces AI-agent application Threat assessment with Attack Graphs (ATAG), a novel framework designed to systematically analyze the security risks associated with AI-agent applications. ATAG extends the MulVAL logic-based AG generation tool with custom facts and interaction rules to accurately represent AI-agent topologies, vulnerabilities, and attack scenarios. As part of this research, we also created the LLM vulnerability database (LVD) to initiate the process of standardizing LLM vulnerabilities documentation. To demonstrate ATAG's efficacy, we applied it to two multi-agent applications. Our case studies demonstrated the framework's ability to model and generate AGs for sophisticated, multi-step attack scenarios exploiting vulnerabilities such as prompt injection, excessive agency, sensitive information disclosure, and insecure output handling across interconnected agents. ATAG is an important step toward a robust methodology and toolset to help understand, visualize, and prioritize complex attack paths in multi-agent AI systems (MAASs). It facilitates proactive identification and mitigation of AI-agent threats in multi-agent applications.