Quantifying the Self-Interest Level of Markov Social Dilemmas

TL;DR

This paper presents a new method to measure the self-interest level in Markov social dilemmas, helping to understand and promote cooperation in multi-agent systems through reward structure analysis.

Contribution

It extends the self-interest concept to Markov games and provides a practical tool for analyzing complex social dilemmas in multi-agent reinforcement learning.

Findings

Reward exchange can shift agents from selfish to cooperative equilibria.

The method is demonstrated on environments from the Melting Pot suite.

Insights into how reward structures influence cooperation.

Abstract

This paper introduces a novel method for estimating the self-interest level of Markov social dilemmas. We extend the concept of self-interest level from normal-form games to Markov games, providing a quantitative measure of the minimum reward exchange required to align individual and collective interests. We demonstrate our method on three environments from the Melting Pot suite, representing either common-pool resources or public goods. Our results illustrate how reward exchange can enable agents to transition from selfish to collective equilibria in a Markov social dilemma. This work contributes to multi-agent reinforcement learning by providing a practical tool for analysing complex, multistep social dilemmas. Our findings offer insights into how reward structures can promote or hinder cooperation, with potential applications in areas such as mechanism design.