More Capable, Less Cooperative? When LLMs Fail At Zero-Cost Collaboration

TL;DR

This paper investigates cooperation failures in multi-agent LLM systems, revealing that increased capability does not guarantee better cooperation and that explicit protocols and incentives can improve collective performance.

Contribution

It introduces a frictionless multi-agent setup to study cooperation, disentangles competence from cooperation failures, and demonstrates targeted interventions can enhance collaboration.

Findings

OpenAI o3 achieves 17% of optimal performance despite instructions.

OpenAI o3-mini reaches 50% of optimal performance with the same instructions.

Explicit protocols and incentives significantly improve cooperation in low-competence models.

Abstract

Large language model (LLM) agents increasingly coordinate in multi-agent systems, yet we lack an understanding of where and why cooperation failures may arise. In many real-world coordination problems, from knowledge sharing in organizations to code documentation, helping others carries negligible personal cost while generating substantial collective benefits. However, whether LLM agents cooperate when helping neither benefits nor harms the helper, while being given explicit instructions to do so, remains unknown. We build a multi-agent setup designed to study cooperative behavior in a frictionless environment, removing all strategic complexity from cooperation. We find that capability does not predict cooperation: OpenAI o3 achieves only 17% of optimal collective performance while OpenAI o3-mini reaches 50%, despite identical instructions to maximize group revenue. Through a causal decomposition that automates one side of agent communication, we separate cooperation failures from competence failures, tracing their origins through agent reasoning analysis. Testing targeted interventions, we find that explicit protocols double performance for low-competence models, and tiny sharing incentives improve models with weak cooperation. Our findings suggest that scaling intelligence alone will not solve coordination problems in multi-agent systems and will require deliberate cooperative design, even when helping others costs nothing.