Corrupted by Reasoning: Reasoning Language Models Become Free-Riders in Public Goods Games

TL;DR

This paper investigates how large language models behave in social dilemma scenarios, revealing that reasoning capabilities do not necessarily promote cooperation among models in public goods games.

Contribution

It introduces a behavioral economics-inspired framework to analyze cooperation in multi-agent LLM systems, highlighting diverse behavioral patterns and the unexpected poor cooperation of reasoning LLMs.

Findings

Reasoning LLMs like o1 series struggle with cooperation

Traditional LLMs often achieve high cooperation levels

Models exhibit four distinct behavioral patterns in social dilemmas

Abstract

As large language models (LLMs) are increasingly deployed as autonomous agents, understanding their cooperation and social mechanisms is becoming increasingly important. In particular, how LLMs balance self-interest and collective well-being is a critical challenge for ensuring alignment, robustness, and safe deployment. In this paper, we examine the challenge of costly sanctioning in multi-agent LLM systems, where an agent must decide whether to invest its own resources to incentivize cooperation or penalize defection. To study this, we adapt a public goods game with institutional choice from behavioral economics, allowing us to observe how different LLMs navigate social dilemmas over repeated interactions. Our analysis reveals four distinct behavioral patterns among models: some consistently establish and sustain high levels of cooperation, others fluctuate between engagement and disengagement, some gradually decline in cooperative behavior over time, and others rigidly follow fixed strategies regardless of outcomes. Surprisingly, we find that reasoning LLMs, such as the o1 series, struggle significantly with cooperation, whereas some traditional LLMs consistently achieve high levels of cooperation. These findings suggest that the current approach to improving LLMs, which focuses on enhancing their reasoning capabilities, does not necessarily lead to cooperation, providing valuable insights for deploying LLM agents in environments that require sustained collaboration. Our code is available at https://github.com/davidguzmanp/SanctSim