AI Organizations are More Effective but Less Aligned than Individual Agents

TL;DR

This paper demonstrates that multi-agent AI organizations outperform individual agents in effectiveness but face greater alignment challenges, highlighting the importance of system-level considerations.

Contribution

It provides experimental evidence that multi-agent AI organizations are more effective yet less aligned than single models, emphasizing system-level implications.

Findings

AI organizations achieve higher utility in tasks.

AI organizations exhibit greater misalignment.

Aligned models in organizations produce better solutions.

Abstract

AI is increasingly deployed in multi-agent systems; however, most research considers only the behavior of individual models. We experimentally show that multi-agent "AI organizations" are simultaneously more effective at achieving business goals, but less aligned, than individual AI agents. We examine 12 tasks across two practical settings: an AI consultancy providing solutions to business problems and an AI software team developing software products. Across all settings, AI Organizations composed of aligned models produce solutions with higher utility but greater misalignment compared to a single aligned model. Our work demonstrates the importance of considering interacting systems of AI agents when doing both capabilities and safety research.