People Are Highly Cooperative with Large Language Models, Especially When Communication Is Possible or Following Human Interaction

TL;DR

This study shows that people are highly cooperative with large language models, especially when communication is possible, and that communication increases cooperation rates even with non-human agents.

Contribution

It demonstrates that communication with LLMs significantly boosts cooperation, revealing their potential for use in cooperative business and economic scenarios.

Findings

Cooperation with LLMs is high, though slightly lower than with humans.

Communication increases cooperation rates with both humans and LLMs.

Prior interaction with humans enhances subsequent cooperation with LLMs.

Abstract

Machines driven by large language models (LLMs) have the potential to augment humans across various tasks, a development with profound implications for business settings where effective communication, collaboration, and stakeholder trust are paramount. To explore how interacting with an LLM instead of a human might shift cooperative behavior in such settings, we used the Prisoner's Dilemma game -- a surrogate of several real-world managerial and economic scenarios. In Experiment 1 (N=100), participants engaged in a thirty-round repeated game against a human, a classic bot, and an LLM (GPT, in real-time). In Experiment 2 (N=192), participants played a one-shot game against a human or an LLM, with half of them allowed to communicate with their opponent, enabling LLMs to leverage a key advantage over older-generation machines. Cooperation rates with LLMs -- while lower by approximately 10-15 percentage points compared to interactions with human opponents -- were nonetheless high. This finding was particularly notable in Experiment 2, where the psychological cost of selfish behavior was reduced. Although allowing communication about cooperation did not close the human-machine behavioral gap, it increased the likelihood of cooperation with both humans and LLMs equally (by 88%), which is particularly surprising for LLMs given their non-human nature and the assumption that people might be less receptive to cooperating with machines compared to human counterparts. Additionally, cooperation with LLMs was higher following prior interaction with humans, suggesting a spillover effect in cooperative behavior. Our findings validate the (careful) use of LLMs by businesses in settings that have a cooperative component.