AC2C: Adaptively Controlled Two-Hop Communication for Multi-Agent Reinforcement Learning

TL;DR

AC2C introduces an adaptive two-hop communication protocol for multi-agent reinforcement learning, enabling efficient long-range information exchange with reduced communication costs, improving performance in dynamic, limited-range communication environments.

Contribution

The paper proposes AC2C, a novel adaptive communication protocol that dynamically manages two-hop messaging to enhance multi-agent RL performance while reducing communication overhead.

Findings

AC2C outperforms baseline methods in cooperative tasks.

AC2C reduces communication costs compared to fully-connected models.

AC2C effectively enables long-range information exchange.

Abstract

Learning communication strategies in cooperative multi-agent reinforcement learning (MARL) has recently attracted intensive attention. Early studies typically assumed a fully-connected communication topology among agents, which induces high communication costs and may not be feasible. Some recent works have developed adaptive communication strategies to reduce communication overhead, but these methods cannot effectively obtain valuable information from agents that are beyond the communication range. In this paper, we consider a realistic communication model where each agent has a limited communication range, and the communication topology dynamically changes. To facilitate effective agent communication, we propose a novel communication protocol called Adaptively Controlled Two-Hop Communication (AC2C). After an initial local communication round, AC2C employs an adaptive two-hop communication strategy to enable long-range information exchange among agents to boost performance, which is implemented by a communication controller. This controller determines whether each agent should ask for two-hop messages and thus helps to reduce the communication overhead during distributed execution. We evaluate AC2C on three cooperative multi-agent tasks, and the experimental results show that it outperforms relevant baselines with lower communication costs.