Emergence of Fair Leaders via Mediators in Multi-Agent Reinforcement Learning

TL;DR

This paper introduces a multi-agent reinforcement learning framework that uses mediators to promote fairness in leader selection within Stackelberg games, addressing issues of bias and self-interest among agents.

Contribution

It formally defines the leader selection problem in multi-agent RL and proposes a mediator-based framework to enhance fairness in agents' outcomes.

Findings

Mediators lead to fairer agent actions and outcomes.

The framework increases overall fairness in agents' returns.

Self-interested agents act more fairly with mediators present.

Abstract

Stackelberg games and their resulting equilibria have received increasing attention in the multi-agent reinforcement learning literature. Each stage of a traditional Stackelberg game involves a leader(s) acting first, followed by the followers. In situations where the roles of leader(s) and followers can be interchanged, the designated role can have considerable advantages, for example, in first-mover advantage settings. Then the question arises: Who should be the leader and when? A bias in the leader selection process can lead to unfair outcomes. This problem is aggravated if the agents are self-interested and care only about their goals and rewards. We formally define this leader selection problem and show its relation to fairness in agents' returns. Furthermore, we propose a multi-agent reinforcement learning framework that maximizes fairness by integrating mediators. Mediators have previously been used in the simultaneous action setting with varying levels of control, such as directly performing agents' actions or just recommending them. Our framework integrates mediators in the Stackelberg setting with minimal control (leader selection). We show that the presence of mediators leads to self-interested agents taking fair actions, resulting in higher overall fairness in agents' returns.