Formal Contracts Mitigate Social Dilemmas in Multi-Agent RL
Andreas A. Haupt, Phillip J.K. Christoffersen, Mehul Damani, Dylan, Hadfield-Menell

TL;DR
This paper introduces formal contracts into multi-agent reinforcement learning to align individual incentives with social welfare, ensuring cooperative behavior and optimal outcomes in various environments.
Contribution
It proposes a contract-based augmentation to Markov games that guarantees socially optimal equilibria and improves welfare, supported by theoretical proofs and empirical experiments.
Findings
Contracts lead to socially optimal equilibria in Markov games.
Richer contract spaces increase overall welfare.
The MOCA training method mitigates exploration issues in contracting environments.
Abstract
Multi-agent Reinforcement Learning (MARL) is a powerful tool for training autonomous agents acting independently in a common environment. However, it can lead to sub-optimal behavior when individual incentives and group incentives diverge. Humans are remarkably capable at solving these social dilemmas. It is an open problem in MARL to replicate such cooperative behaviors in selfish agents. In this work, we draw upon the idea of formal contracting from economics to overcome diverging incentives between agents in MARL. We propose an augmentation to a Markov game where agents voluntarily agree to binding transfers of reward, under pre-specified conditions. Our contributions are theoretical and empirical. First, we show that this augmentation makes all subgame-perfect equilibria of all Fully Observable Markov Games exhibit socially optimal behavior, given a sufficiently rich space of…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMulti-Agent Systems and Negotiation
