Learning enables adaptation in cooperation for multi-player stochastic games

TL;DR

This paper develops a game-theoretical framework using reinforcement learning to understand how individuals adapt their cooperative behaviors in changing environments within multi-player stochastic games, revealing environment-mediated effects on cooperation.

Contribution

It introduces a novel analytical model combining evolutionary game theory and reinforcement learning to study cooperation in dynamic environments, highlighting the role of learning in social dilemmas.

Findings

Learning enhances cooperation in weak social dilemmas.

Learning hinders cooperation in strong social dilemmas.

Environmental fluctuations influence the evolution of cooperation.

Abstract

Interactions among individuals in natural populations often occur in a dynamically changing environment. Understanding the role of environmental variation in population dynamics has long been a central topic in theoretical ecology and population biology. However, the key question of how individuals, in the middle of challenging social dilemmas (e.g., the "tragedy of the commons"), modulate their behaviors to adapt to the fluctuation of the environment has not yet been addressed satisfactorily. Utilizing evolutionary game theory and stochastic games, we develop a game-theoretical framework that incorporates the adaptive mechanism of reinforcement learning to investigate whether cooperative behaviors can evolve in the ever-changing group interaction environment. When the action choices of players are just slightly influenced by past reinforcements, we construct an analytical condition to determine whether cooperation can be favored over defection. Intuitively, this condition reveals why and how the environment can mediate cooperative dilemmas. Under our model architecture, we also compare this learning mechanism with two non-learning decision rules, and we find that learning significantly improves the propensity for cooperation in weak social dilemmas, and, in sharp contrast, hinders cooperation in strong social dilemmas. Our results suggest that in complex social-ecological dilemmas, learning enables the adaptation of individuals to varying environments.