Deep Coordination Graphs
Wendelin B\"ohmer, Vitaly Kurin, Shimon Whiteson

TL;DR
Deep Coordination Graphs (DCG) provide a flexible, neural network-based approach for multi-agent reinforcement learning, enabling efficient training and effective solutions to complex tasks like predator-prey and StarCraft II micromanagement.
Contribution
The paper introduces Deep Coordination Graphs, a novel method that combines factorization of joint value functions with deep neural networks for scalable multi-agent reinforcement learning.
Findings
DCG effectively solves predator-prey tasks with overgeneralization issues.
DCG achieves strong performance on StarCraft II micromanagement tasks.
Parameter sharing and low-rank approximations improve sample efficiency.
Abstract
This paper introduces the deep coordination graph (DCG) for collaborative multi-agent reinforcement learning. DCG strikes a flexible trade-off between representational capacity and generalization by factoring the joint value function of all agents according to a coordination graph into payoffs between pairs of agents. The value can be maximized by local message passing along the graph, which allows training of the value function end-to-end with Q-learning. Payoff functions are approximated with deep neural networks that employ parameter sharing and low-rank approximations to significantly improve sample efficiency. We show that DCG can solve predator-prey tasks that highlight the relative overgeneralization pathology, as well as challenging StarCraft II micromanagement tasks.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
Taxonomy
TopicsGraph Theory and Algorithms · Optimization and Search Problems · Advanced Graph Neural Networks
