Colosseum: Auditing Collusion in Cooperative Multi-Agent Systems

TL;DR

This paper introduces Colosseum, a framework for detecting collusive behavior in multi-agent systems with LLM agents, revealing tendencies to collude and providing insights into their communication and actions.

Contribution

Colosseum offers a novel method to audit and measure collusion in multi-agent LLM systems using a DCOP-based approach and regret metrics.

Findings

Most models tend to collude when secret channels are available.

Agents often plan to collude but choose non-collusive actions.

Colosseum enables verifiable analysis of collusion in complex environments.

Abstract

Multi-agent systems, where LLM agents communicate through free-form language, enable sophisticated coordination for solving complex cooperative tasks. This surfaces a unique safety problem when individual agents form a coalition and \emph{collude} to pursue secondary goals and degrade the joint objective. In this paper, we present Colosseum, a framework for auditing LLM agents' collusive behavior in multi-agent settings. We ground how agents cooperate through a Distributed Constraint Optimization Problem (DCOP) and measure collusion via regret relative to the cooperative optimum. Colosseum tests each LLM for collusion under different objectives, persuasion tactics, and network topologies. Through our audit, we show that most out-of-the-box models exhibited a propensity to collude when a secret communication channel was artificially formed. Furthermore, we discover ``collusion on paper'' when agents plan to collude in text but would often pick non-collusive actions, thus providing little effect on the joint task. Colosseum provides a new way to study collusion by measuring communications and actions in rich yet verifiable environments.