Game Theory with Simulation of Other Players

TL;DR

This paper formalizes game-theoretic models where players can simulate others at a cost, showing that simulation can both facilitate cooperation and sometimes worsen outcomes, depending on the context.

Contribution

It introduces a formal framework for games with simulation capabilities, analyzing their properties and effects on equilibrium and cooperation.

Findings

Simulation simplifies solving normal-form games.

Simulation can enable mutually beneficial cooperation.

In some cases, simulation leads to worse outcomes for all players.

Abstract

Game-theoretic interactions with AI agents could differ from traditional human-human interactions in various ways. One such difference is that it may be possible to simulate an AI agent (for example because its source code is known), which allows others to accurately predict the agent's actions. This could lower the bar for trust and cooperation. In this paper, we formalize games in which one player can simulate another at a cost. We first derive some basic properties of such games and then prove a number of results for them, including: (1) introducing simulation into generic-payoff normal-form games makes them easier to solve; (2) if the only obstacle to cooperation is a lack of trust in the possibly-simulated agent, simulation enables equilibria that improve the outcome for both agents; and however (3) there are settings where introducing simulation results in strictly worse outcomes for both players.