The Good, the Bad, and the Hulk-like GPT: Analyzing Emotional Decisions of Large Language Models in Cooperation and Bargaining Games

TL;DR

This paper explores how emotions influence the decision-making of large language models like GPT-3.5 and GPT-4 in behavioral games, revealing that emotions can significantly alter their strategies and alignment with human behavior.

Contribution

It introduces a novel framework to study emotional impacts on LLM decision-making and compares their behavior to humans in game-theoretic scenarios.

Findings

Emotions significantly affect LLM strategies in behavioral games.

GPT-3.5 aligns closely with human responses, especially in bargaining.

GPT-4's rational behavior is disrupted by emotional prompts, notably anger.

Abstract

Behavior study experiments are an important part of society modeling and understanding human interactions. In practice, many behavioral experiments encounter challenges related to internal and external validity, reproducibility, and social bias due to the complexity of social interactions and cooperation in human user studies. Recent advances in Large Language Models (LLMs) have provided researchers with a new promising tool for the simulation of human behavior. However, existing LLM-based simulations operate under the unproven hypothesis that LLM agents behave similarly to humans as well as ignore a crucial factor in human decision-making: emotions. In this paper, we introduce a novel methodology and the framework to study both, the decision-making of LLMs and their alignment with human behavior under emotional states. Experiments with GPT-3.5 and GPT-4 on four games from two different classes of behavioral game theory showed that emotions profoundly impact the performance of LLMs, leading to the development of more optimal strategies. While there is a strong alignment between the behavioral responses of GPT-3.5 and human participants, particularly evident in bargaining games, GPT-4 exhibits consistent behavior, ignoring induced emotions for rationality decisions. Surprisingly, emotional prompting, particularly with `anger' emotion, can disrupt the "superhuman" alignment of GPT-4, resembling human emotional responses.