An analytical model of active inference in the Iterated Prisoner's Dilemma

TL;DR

This paper develops a mathematically tractable active inference model for the Iterated Prisoner's Dilemma, analyzing how Bayesian agents' beliefs and learning rates influence game dynamics and steady states.

Contribution

It introduces a Bayesian active inference framework for the Iterated Prisoner's Dilemma, enabling rigorous analysis of phase transitions and steady states in two-agent interactions.

Findings

Critical phase transition points are linearly related to learning rate and reward function.

Varying learning rates affects emergent patterns and game dynamics.

Comparison of stochastic and deterministic solutions reveals key behavioral differences.

Abstract

This paper addresses a mathematically tractable model of the Prisoner's Dilemma using the framework of active inference. In this work, we design pairs of Bayesian agents that are tracking the joint game state of their and their opponent's choices in an Iterated Prisoner's Dilemma game. The specification of the agents' belief architecture in the form of a partially-observed Markov decision process allows careful and rigourous investigation into the dynamics of two-player gameplay, including the derivation of optimal conditions for phase transitions that are required to achieve certain game-theoretic steady states. We show that the critical time points governing the phase transition are linearly related to each other as a function of learning rate and the reward function. We then investigate the patterns that emerge when varying the agents' learning rates, as well as the relationship between the stochastic and deterministic solutions to the two-agent system.