CGIBNet: Bandwidth-constrained Communication with Graph Information Bottleneck in Multi-Agent Reinforcement Learning

TL;DR

This paper introduces CGIBNet, a novel communication model for multi-agent reinforcement learning that compresses both communication structure and content to operate efficiently under bandwidth constraints, improving performance across various frameworks and environments.

Contribution

The paper presents CGIBNet, the first universal module that optimizes communication structure and content compression in bandwidth-limited multi-agent reinforcement learning.

Findings

Outperforms state-of-the-art algorithms in bandwidth-constrained scenarios.

Effective in both policy-based and value-based MARL frameworks.

Applicable to single-round and multi-round communication modes.

Abstract

Communication is one of the core components for cooperative multi-agent reinforcement learning (MARL). The communication bandwidth, in many real applications, is always subject to certain constraints. To improve communication efficiency, in this article, we propose to simultaneously optimize whom to communicate with and what to communicate for each agent in MARL. By initiating the communication between agents with a directed complete graph, we propose a novel communication model, named Communicative Graph Information Bottleneck Network (CGIBNet), to simultaneously compress the graph structure and the node information with the graph information bottleneck principle. The graph structure compression is designed to cut the redundant edges for determining whom to communicate with. The node information compression aims to address the problem of what to communicate via learning compact node representations. Moreover, CGIBNet is the first universal module for bandwidth-constrained communication, which can be applied to various training frameworks (i.e., policy-based and value-based MARL frameworks) and communication modes (i.e., single-round and multi-round communication). Extensive experiments are conducted in Traffic Control and StarCraft II environments. The results indicate that our method can achieve better performance in bandwidth-constrained settings compared with state-of-the-art algorithms.