Evaluating Counterfactual Strategic Reasoning in Large Language Models

TL;DR

This paper assesses whether large language models genuinely reason strategically or rely on memorized patterns by testing them in canonical game-theoretic scenarios with counterfactual modifications.

Contribution

It introduces a multi-metric evaluation framework to analyze LLMs' strategic reasoning and incentive sensitivity in altered game environments.

Findings

LLMs show limited incentive sensitivity in counterfactual settings

Structural generalization in LLMs is constrained

Strategic reasoning capabilities are limited in counterfactual environments

Abstract

We evaluate Large Language Models (LLMs) in repeated game-theoretic settings to assess whether strategic performance reflects genuine reasoning or reliance on memorized patterns. We consider two canonical games, Prisoner's Dilemma (PD) and Rock-Paper-Scissors (RPS), upon which we introduce counterfactual variants that alter payoff structures and action labels, breaking familiar symmetries and dominance relations. Our multi-metric evaluation framework compares default and counterfactual instantiations, showcasing LLM limitations in incentive sensitivity, structural generalization and strategic reasoning within counterfactual environments.