Multi-Agent Security Tax: Trading Off Security and Collaboration Capabilities in Multi-Agent Systems

TL;DR

This paper investigates security risks in multi-agent AI systems, demonstrating how malicious attacks can spread and affect collaboration, and evaluates strategies to mitigate these risks while highlighting security-collaboration trade-offs.

Contribution

The study introduces simulation-based analysis of security vulnerabilities in multi-agent systems and evaluates defense strategies, revealing inherent trade-offs between security and collaboration.

Findings

Malicious prompts can spread through multi-hop in agent networks.

Defense strategies reduce malicious instruction spread but impair collaboration.

Security measures often decrease the efficiency of agent collaboration.

Abstract

As AI agents are increasingly adopted to collaborate on complex objectives, ensuring the security of autonomous multi-agent systems becomes crucial. We develop simulations of agents collaborating on shared objectives to study these security risks and security trade-offs. We focus on scenarios where an attacker compromises one agent, using it to steer the entire system toward misaligned outcomes by corrupting other agents. In this context, we observe infectious malicious prompts - the multi-hop spreading of malicious instructions. To mitigate this risk, we evaluated several strategies: two "vaccination" approaches that insert false memories of safely handling malicious input into the agents' memory stream, and two versions of a generic safety instruction strategy. While these defenses reduce the spread and fulfillment of malicious instructions in our experiments, they tend to decrease collaboration capability in the agent network. Our findings illustrate potential trade-off between security and collaborative efficiency in multi-agent systems, providing insights for designing more secure yet effective AI collaborations.