Stochastic evolution in populations of ideas

TL;DR

This paper models reinforcement learning as a stochastic evolutionary process in finite populations of ideas, highlighting how ideas can go extinct or become fixed, differing from traditional mutation-selection models.

Contribution

It introduces a stochastic birth-death framework for learning in populations of ideas, capturing extinction and fixation phenomena in finite settings.

Findings

Extinction and fixation of ideas are possible in the model.

The dynamics differ from classical mutation-selection processes.

Results are characterized for various symmetric and asymmetric games.

Abstract

It is known that learning of players who interact in a repeated game can be interpreted as an evolutionary process in a population of ideas. These analogies have so far mostly been established in deterministic models, and memory loss in learning has been seen to act similarly to mutation in evolution. We here propose a representation of reinforcement learning as a stochastic process in finite "populations of ideas". The resulting birth-death dynamics has absorbing states and allows for the extinction or fixation of ideas, marking a key difference to mutation-selection processes in finite populations. We characterize the outcome of evolution in populations of ideas for several classes of symmetric and asymmetric games.