Fixation and escape times in stochastic game learning

TL;DR

This paper explores fixation and escape times in stochastic game learning, showing how finite observation batches influence strategy fixation in simple 2x2 games and comparing these phenomena to evolutionary dynamics.

Contribution

It introduces the concept of fixation in stochastic game learning with finite batches, providing analytical and numerical insights into escape times and their dependence on batch size.

Findings

Fixation occurs in stochastic game learning similar to evolutionary dynamics.

Escape times depend on batch size and can be estimated analytically.

Numerical simulations confirm theoretical predictions.

Abstract

Evolutionary dynamics in finite populations is known to fixate eventually in the absence of mutation. We here show that a similar phenomenon can be found in stochastic game dynamical batch learning, and investigate fixation in learning processes in a simple 2x2 game, for two-player games with cyclic interaction, and in the context of the best-shot network game. The analogues of finite populations in evolution are here finite batches of observations between strategy updates. We study when and how such fixation can occur, and present results on the average time-to-fixation from numerical simulations. Simple cases are also amenable to analytical approaches and we provide estimates of the behaviour of so-called escape times as a function of the batch size. The differences and similarities with escape and fixation in evolutionary dynamics are discussed.