The State-Action-Reward-State-Action Algorithm in Spatial Prisoner's Dilemma Game

TL;DR

This paper explores how the SARSA reinforcement learning algorithm influences cooperation dynamics in the spatial Prisoner's Dilemma, demonstrating its effects on agent behavior and network cooperation levels.

Contribution

It introduces the application of SARSA to evolutionary game theory, specifically in modeling agent decision-making and cooperation emergence in spatial settings.

Findings

SARSA affects cooperation rates among agents

Behavioral changes depend on reward structures

Cooperation levels vary with network configurations

Abstract

Cooperative behavior is prevalent in both human society and nature. Understanding the emergence and maintenance of cooperation among self-interested individuals remains a significant challenge in evolutionary biology and social sciences. Reinforcement learning (RL) provides a suitable framework for studying evolutionary game theory as it can adapt to environmental changes and maximize expected benefits. In this study, we employ the State-Action-Reward-State-Action (SARSA) algorithm as the decision-making mechanism for individuals in evolutionary game theory. Initially, we apply SARSA to imitation learning, where agents select neighbors to imitate based on rewards. This approach allows us to observe behavioral changes in agents without independent decision-making abilities. Subsequently, SARSA is utilized for primary agents to independently choose cooperation or betrayal with their neighbors. We evaluate the impact of SARSA on cooperation rates by analyzing variations in rewards and the distribution of cooperators and defectors within the network.