SACHI: Structured Agent Coordination via Holistic Information Integration in Multi-Agent Reinforcement Learning

TL;DR

SACHI introduces a graph transformer-based method for multi-agent reinforcement learning that enhances coordination by integrating holistic information, outperforming existing approaches across diverse tasks.

Contribution

The paper proposes a novel structured information integration approach using graph transformers for multi-agent coordination, demonstrating significant performance improvements.

Findings

SACHI outperforms 12 baselines on five cooperative tasks.

Statistical analyses confirm the significance and robustness of results.

Content-dependent message passing is key to performance gains.

Abstract

Cooperative multi-agent reinforcement learning agents that act on partial local observations face a fundamental information bottleneck: the knowledge needed to select jointly optimal actions is scattered across the team, yet each agent must commit to a decision without access to its teammates' observations, intentions, or chosen actions. Existing methods either ignore this bottleneck, compress it into a scalar mixing signal, or route around it with learned communication channels. Framing action coordination as a problem of structured information integration among agents, we propose \textit{structured agent coordination via holistic information integration}, or SACHI, in which graph transformer convolutions over an inter-agent coordination graph enrich each agent's representation with receiver-sensitive, content-dependent signals from teammates prior to action selection. We evaluate SACHI across five cooperative tasks spanning spatial, communicative, and adversarial coordination challenges against twelve baselines. SACHI consistently matches or outperforms the best baseline on every task, and rigorous aggregate statistical analyses, including normalized metrics with bootstrap confidence intervals, Friedman ranking, and performance profiling, confirm that this advantage is statistically significant, robust across environments, and not attributable to increased model capacity. Parameter-matched ablations further trace the source of the gains to a single architectural property: the degree of content-dependence in the message-passing operator.