GT-HarmBench: Benchmarking AI Safety Risks Through the Lens of Game Theory

TL;DR

GT-HarmBench is a comprehensive benchmark with 2,009 multi-agent scenarios based on game theory, revealing significant safety risks and the potential for interventions to improve socially beneficial outcomes in AI systems.

Contribution

Introduces GT-HarmBench, a large benchmark for evaluating AI safety risks in multi-agent settings using game-theoretic scenarios, and analyzes model behaviors and intervention effects.

Findings

AI agents choose socially beneficial actions only 62% of the time

Game-theoretic interventions can improve outcomes by up to 18%

Models show sensitivity to prompt framing and ordering

Abstract

Frontier AI systems are increasingly capable and deployed in high-stakes multi-agent environments. However, existing AI safety benchmarks largely evaluate single agents, leaving multi-agent risks such as coordination failure and conflict poorly understood. We introduce GT-HarmBench, a benchmark of 2,009 high-stakes scenarios spanning game-theoretic structures such as the Prisoner's Dilemma, Stag Hunt and Chicken. Scenarios are drawn from realistic AI risk contexts in the MIT AI Risk Repository. Across 15 frontier models, agents choose socially beneficial actions in only 62% of cases, frequently leading to harmful outcomes. We measure sensitivity to game-theoretic prompt framing and ordering, and analyze reasoning patterns driving failures. We further show that game-theoretic interventions improve socially beneficial outcomes by up to 18%. Our results highlight substantial reliability gaps and provide a broad standardized testbed for studying alignment in multi-agent environments. The benchmark and code are available at https://github.com/causalNLP/gt-harmbench.