Dynamic Incentivized Cooperation under Changing Rewards

TL;DR

This paper introduces DRIVE, an adaptive peer incentivization method that maintains cooperation in social dilemmas despite changing environmental rewards, addressing limitations of fixed incentive approaches.

Contribution

The paper presents DRIVE, a novel decentralized incentive mechanism that dynamically adjusts rewards to sustain cooperation under variable environmental conditions.

Findings

DRIVE successfully maintains cooperation in the Prisoner's Dilemma.

Empirical results show DRIVE outperforms fixed-incentive methods in changing reward scenarios.

DRIVE adapts to complex sequential social dilemmas with changing rewards.

Abstract

Peer incentivization (PI) is a popular multi-agent reinforcement learning approach where all agents can reward or penalize each other to achieve cooperation in social dilemmas. Despite their potential for scalable cooperation, current PI methods heavily depend on fixed incentive values that need to be appropriately chosen with respect to the environmental rewards and thus are highly sensitive to their changes. Therefore, they fail to maintain cooperation under changing rewards in the environment, e.g., caused by modified specifications, varying supply and demand, or sensory flaws - even when the conditions for mutual cooperation remain the same. In this paper, we propose Dynamic Reward Incentives for Variable Exchange (DRIVE), an adaptive PI approach to cooperation in social dilemmas with changing rewards. DRIVE agents reciprocally exchange reward differences to incentivize mutual cooperation in a completely decentralized way. We show how DRIVE achieves mutual cooperation in the general Prisoner's Dilemma and empirically evaluate DRIVE in more complex sequential social dilemmas with changing rewards, demonstrating its ability to achieve and maintain cooperation, in contrast to current state-of-the-art PI methods.