Heterogeneous Information-Bottleneck Coordination Graphs for Multi-Agent Reinforcement Learning

TL;DR

This paper introduces Heterogeneous Information-Bottleneck Coordination Graphs (HIBCG), a theoretically grounded method for learning sparse, task-relevant communication structures in multi-agent reinforcement learning.

Contribution

HIBCG provides a principled approach to learn agent communication topologies with controlled message capacities, improving over heuristic methods.

Findings

The group-aligned prior tightens the variational bound on topology learning.

The method enables differential edge control per group block.

Capacity allocation follows a water-filling principle.

Abstract

Coordination graphs are a central abstraction in cooperative multi-agent reinforcement learning (MARL), yet existing sparse-graph learners lack a theoretically grounded mechanism to decide which edges should exist and how much information each edge should carry. Current methods rely on heuristic criteria that offer no formal guarantee on the learned topology, and no principled way to allocate different communication capacities to structurally different agent relationships. To address this, we propose Heterogeneous Information-Bottleneck Coordination Graphs (HIBCG), which learns a group-aware sparse graph in which both edge existence and message capacity are theoretically justified. With the graph information bottleneck (GIB) serving as the underlying tool, HIBCG first constructs a group-aligned block-diagonal prior that provides a closed-form criterion for edge retention -- determining which edges should exist and at what density per group block -- and then controls per-agent feature bandwidth on the resulting topology, compressing messages to retain only task-relevant content. We prove that the group-aligned prior strictly tightens the variational bound on topology learning, that the objective decomposes per group block, enabling differential edge control, and that capacity allocation follows a water-filling principle.