Strategies for updating rules driven by reinforcement learning to solve social dilemmas

TL;DR

This paper introduces a new strategy update rule in reinforcement learning that improves cooperation in social dilemmas by incorporating historical performance.

Contribution

The novelty lies in integrating historical performance into imitation rules, enhancing cooperation through a moderated update mechanism.

Findings

The proposed strategy update rule promotes cooperation more effectively than traditional methods.

Increasing the parameter δ further enhances systemic cooperation.

Multidimensional evaluation mechanisms help explain cooperative behavior in complex environments.

Abstract

This study incorporates historical performance into traditional imitation rules and proposes a moderated strategy update rule. In this framework, an individual’s temporal historical performance is calculated using the BM model. By adjusting the parameter δ, the influence of historical performance on strategy learning is determined, and the evolution of cooperation is subsequently observed. Results show that the proposed strategy update rule promotes cooperation more effectively than the traditional version, and systemic cooperation is further enhanced as δ increases. The reason why the proposed rule enhances cooperation is that it amplifies the evaluation of cooperative behavior while compressing the evaluation of defective behavior. Although establishing system objectives may hinder the diffusion of cooperative behavior, appropriate performance evaluation mechanisms can mitigate this adverse effect. Our results indicate that multidimensional evaluation can provide a theoretical basis for explaining cooperative behavior in complex environments.

Figures