Altruism and Fair Objective in Mixed-Motive Markov games

TL;DR

This paper introduces a novel framework for promoting fairer cooperation in social dilemmas by replacing utilitarian objectives with Proportional Fairness, and develops algorithms for sequential settings to learn fair policies.

Contribution

It proposes a new fair altruistic utility concept, extends it to Markov games, and derives algorithms for fair policy learning in social dilemmas.

Findings

The framework ensures cooperation under social dilemmas.

Fair policies outperform utilitarian ones in fairness metrics.

Algorithms successfully learn fair strategies in various environments.

Abstract

Cooperation is fundamental for society's viability, as it enables the emergence of structure within heterogeneous groups that seek collective well-being. However, individuals are inclined to defect in order to benefit from the group's cooperation without contributing the associated costs, thus leading to unfair situations. In game theory, social dilemmas entail this dichotomy between individual interest and collective outcome. The most dominant approach to multi-agent cooperation is the utilitarian welfare which can produce efficient highly inequitable outcomes. This paper proposes a novel framework to foster fairer cooperation by replacing the standard utilitarian objective with Proportional Fairness. We introduce a fair altruistic utility for each agent, defined on the individual log-payoff space and derive the analytical conditions required to ensure cooperation in classic social dilemmas. We then extend this framework to sequential settings by defining a Fair Markov Game and deriving novel fair Actor-Critic algorithms to learn fair policies. Finally, we evaluate our method in various social dilemma environments.