Fairness over Equality: Correcting Social Incentives in Asymmetric Sequential Social Dilemmas

TL;DR

This paper investigates how social incentives and fairness mechanisms in multi-agent reinforcement learning can be adapted to asymmetric environments, proposing modifications that improve cooperation under real-world social and informational constraints.

Contribution

The paper introduces asymmetric variants of SSD environments and proposes three modifications to fairness-based methods to better handle asymmetries and partial observability.

Findings

Existing fairness methods struggle with asymmetries, often incentivizing defection.

Proposed modifications improve cooperation speed in asymmetric scenarios.

Methods remain scalable and practical under partial observability.

Abstract

Sequential Social Dilemmas (SSDs) provide a key framework for studying how cooperation emerges when individual incentives conflict with collective welfare. In Multi-Agent Reinforcement Learning, these problems are often addressed by incorporating intrinsic drives that encourage prosocial or fair behavior. However, most existing methods assume that agents face identical incentives in the dilemma and require continuous access to global information about other agents to assess fairness. In this work, we introduce asymmetric variants of well-known SSD environments and examine how natural differences between agents influence cooperation dynamics. Our findings reveal that existing fairness-based methods struggle to adapt under asymmetric conditions by enforcing raw equality that wrongfully incentivize defection. To address this, we propose three modifications: (i) redefining fairness by accounting for agents' reward ranges, (ii) introducing an agent-based weighting mechanism to better handle inherent asymmetries, and (iii) localizing social feedback to make the methods effective under partial observability without requiring global information sharing. Experimental results show that in asymmetric scenarios, our method fosters faster emergence of cooperative policies compared to existing approaches, without sacrificing scalability or practicality.