Reinforcement learning with reputation-based adaptive exploration promotes the evolution of cooperation

TL;DR

This paper introduces a reputation-based adaptive exploration mechanism in multi-agent reinforcement learning, which promotes cooperation by adjusting exploration based on social reputation and status.

Contribution

It proposes a novel Q-learning model coupling exploration rates with reputation differences and asymmetric reputation updates, enhancing cooperation in social environments.

Findings

Each mechanism independently promotes cooperation.

Combining mechanisms yields a reinforcing effect on cooperation.

Reputation-based adjustments improve learning outcomes in social contexts.

Abstract

Multi-agent reinforcement learning serves as an effective tool for studying strategy adaptation in evolutionary games. Although prior work has integrated Q-learning with reputation mechanisms to promote cooperation, most existing algorithms adopt fixed exploration rates and overlook the influence of social context on exploratory behavior. In practice, individuals may adjust their willingness to explore based on their reputation and perceived social standing. To address this, we propose a Q-learning model that couples exploration rates with local reputation differences and incorporates asymmetric, state-dependent reputation updates. Our results show that each mechanism independently promotes cooperation, and their combination yields a reinforcing effect. The joint mechanism enhances cooperation by making ``high reputation--low exploration, low reputation--high exploration'', while adjusting reputation updates to amplify cooperative gains at low status and defection penalties at high status. This study thus offers insights into how social evaluation can shape learning behavior in complex environments.