TACTIC: Task-Agnostic Contrastive pre-Training for Inter-Agent Communication

TL;DR

TACTIC introduces a task-agnostic contrastive pre-training approach for inter-agent communication in multi-agent reinforcement learning, significantly improving generalization across different sight ranges and enhancing team coordination.

Contribution

It proposes a novel adaptive communication mechanism with contrastive learning that generalizes to unseen sight ranges, addressing a key challenge in cooperative MARL.

Findings

Outperforms state-of-the-art MARL methods on SMACv2 benchmark

Enhances coordination in scenarios with limited or extensive observability

Demonstrates strong generalization to unseen sight ranges

Abstract

The "sight range dilemma" in cooperative Multi-Agent Reinforcement Learning (MARL) presents a significant challenge: limited observability hinders team coordination, while extensive sight ranges lead to distracted attention and reduced performance. While communication can potentially address this issue, existing methods often struggle to generalize across different sight ranges, limiting their effectiveness. We propose TACTIC, Task-Agnostic Contrastive pre-Training strategy Inter-Agent Communication. TACTIC is an adaptive communication mechanism that enhances agent coordination even when the sight range during execution is vastly different from that during training. The communication mechanism encodes messages and integrates them with local observations, generating representations grounded in the global state using contrastive learning. By learning to generate and interpret messages that capture important information about the whole environment, TACTIC enables agents to effectively "see" more through communication, regardless of their sight ranges. We comprehensively evaluate TACTIC on the SMACv2 benchmark across various scenarios with broad sight ranges. The results demonstrate that TACTIC consistently outperforms traditional state-of-the-art MARL techniques with and without communication, in terms of generalizing to sight ranges different from those seen in training, particularly in cases of extremely limited or extensive observability.