Dilution, Diffusion and Symbiosis in Spatial Prisoner's Dilemma with Reinforcement Learning

TL;DR

This paper investigates how dilution and mobility influence cooperation in spatial prisoner's dilemma games using multi-agent Q-learning, revealing effects like equivalence of fixed and learned update rules and emergent mutualism.

Contribution

It introduces a multi-action Q-learning framework for spatial prisoner's dilemma, exploring effects of dilution and mobility, and demonstrates the emergence of symbiosis and equivalence with fixed rules.

Findings

Games with fixed update rules can be qualitatively similar to learned ones.

Emergence of mutualistic effects between populations with multiple actions.

Versatility of the Q-learning approach in modeling diverse game scenarios.

Abstract

Recent studies in the spatial prisoner's dilemma games with reinforcement learning have shown that static agents can learn to cooperate through a diverse sort of mechanisms, including noise injection, different types of learning algorithms and neighbours' payoff knowledge. In this work, using an independent multi-agent Q-learning algorithm, we study the effects of dilution and mobility in the spatial version of the prisoner's dilemma. Within this setting, different possible actions for the algorithm are defined, connecting with previous results on the classical, non-reinforcement learning spatial prisoner's dilemma, showcasing the versatility of the algorithm in modeling different game-theoretical scenarios and the benchmarking potential of this approach. As a result, a range of effects is observed, including evidence that games with fixed update rules can be qualitatively equivalent to those with learned ones, as well as the emergence of a symbiotic mutualistic effect between populations that forms when multiple actions are defined.