Learning Individual Policies in Large Multi-agent Systems through Local Variance Minimization

TL;DR

This paper introduces a novel multi-agent reinforcement learning mechanism for large systems, minimizing variance among agents' values to improve fairness and revenue, demonstrated on taxi data and simulations.

Contribution

The paper proposes a new MARL mechanism based on variance minimization in large multi-agent systems with nearly non-atomic agents, enhancing fairness and revenue.

Findings

Reduces revenue variance among agents

Achieves higher joint revenues than existing methods

Effective on real-world taxi data and simulations

Abstract

In multi-agent systems with large number of agents, typically the contribution of each agent to the value of other agents is minimal (e.g., aggregation systems such as Uber, Deliveroo). In this paper, we consider such multi-agent systems where each agent is self-interested and takes a sequence of decisions and represent them as a Stochastic Non-atomic Congestion Game (SNCG). We derive key properties for equilibrium solutions in SNCG model with non-atomic and also nearly non-atomic agents. With those key equilibrium properties, we provide a novel Multi-Agent Reinforcement Learning (MARL) mechanism that minimizes variance across values of agents in the same state. To demonstrate the utility of this new mechanism, we provide detailed results on a real-world taxi dataset and also a generic simulator for aggregation systems. We show that our approach reduces the variance in revenues earned by taxi drivers, while still providing higher joint revenues than leading approaches.