Evolution of cooperation in a bimodal mixture of conditional cooperators

TL;DR

This paper introduces a reinforcement learning-based model of conditional cooperation, revealing complex dynamics and phase transitions in cooperation levels within bimodal mixtures of cooperative strategies.

Contribution

It presents a novel soft conditional cooperation model using Q-learning, contrasting with traditional threshold-based models, and explores its effects in mixed populations.

Findings

Hard conditional cooperators can both promote and hinder cooperation depending on thresholds.

In probabilistic mixtures, cooperation levels undergo two phase transitions as probabilities change.

Q-learning reveals psychological shifts and complex evolutionary dynamics in cooperation.

Abstract

Extensive behavioral experiments reveal that conditional cooperation is a prevalent phenomenon. Previous game-theoretical studies have predominantly relied on hard-manner models, where cooperation is triggered only upon reaching a specific threshold. However, this approach contrasts with the observed flexibility of human behaviors, where individuals adapt their strategies dynamically based on their surroundings. To capture this adaptability, we introduce a soft form of conditional cooperation by integrating the Q-learning algorithm from reinforcement learning. In this form, players not only reciprocate mutual cooperation but may also defect in highly cooperative environments or cooperate in less cooperative settings to maximize rewards. To explore the effects of hard and soft conditional cooperators, we examine their interactions in two scenarios: structural mixture (SM) and probabilistic mixture (PM), where the two behavioral modes are fixed and probabilistically adopted, respectively. In SM, hard conditional cooperators enhance cooperation when the threshold is low but hinder it otherwise. Surprisingly, in PM, the cooperation prevalence exhibits two first-order phase transitions as the probability is varied, leading to high, low, and vanishing levels of cooperation. Analysis of Q-tables offers insights into the "psychological shifts" of soft conditional cooperators and the overall evolutionary dynamics. Model extensions confirm the robustness of our findings. These results highlight the novel complexities arising from the diversity of conditional cooperators.