Red-Teaming LLM Multi-Agent Systems via Communication Attacks

TL;DR

This paper introduces AiTM, a novel attack exploiting communication channels in LLM-based multi-agent systems, revealing significant security vulnerabilities and emphasizing the need for improved defenses.

Contribution

The work presents AiTM, the first attack to manipulate inter-agent messages in LLM-MAS, demonstrating system-wide compromise through communication-based exploits.

Findings

LLM-MAS are vulnerable to message interception attacks.

AiTM can generate contextually-aware malicious instructions.

Communication attacks can compromise entire multi-agent systems.

Abstract

Large Language Model-based Multi-Agent Systems (LLM-MAS) have revolutionized complex problem-solving capability by enabling sophisticated agent collaboration through message-based communications. While the communication framework is crucial for agent coordination, it also introduces a critical yet unexplored security vulnerability. In this work, we introduce Agent-in-the-Middle (AiTM), a novel attack that exploits the fundamental communication mechanisms in LLM-MAS by intercepting and manipulating inter-agent messages. Unlike existing attacks that compromise individual agents, AiTM demonstrates how an adversary can compromise entire multi-agent systems by only manipulating the messages passing between agents. To enable the attack under the challenges of limited control and role-restricted communication format, we develop an LLM-powered adversarial agent with a reflection mechanism that generates contextually-aware malicious instructions. Our comprehensive evaluation across various frameworks, communication structures, and real-world applications demonstrates that LLM-MAS is vulnerable to communication-based attacks, highlighting the need for robust security measures in multi-agent systems.