Multi-Agent Coordination via Multi-Level Communication

TL;DR

This paper introduces SeqComm, a multi-level asynchronous communication scheme for multi-agent systems that improves coordination by structuring decision-making and communication phases, with proven convergence and superior empirical performance.

Contribution

The paper proposes a novel multi-level communication framework, SeqComm, that addresses circular dependencies in multi-agent coordination through asynchronous decision and communication phases.

Findings

SeqComm guarantees monotonic policy improvement and convergence.

SeqComm outperforms existing methods in cooperative multi-agent tasks.

Theoretical analysis confirms the effectiveness of the proposed scheme.

Abstract

The partial observability and stochasticity in multi-agent settings can be mitigated by accessing more information about others via communication. However, the coordination problem still exists since agents cannot communicate actual actions with each other at the same time due to the circular dependencies. In this paper, we propose a novel multi-level communication scheme, Sequential Communication (SeqComm). SeqComm treats agents asynchronously (the upper-level agents make decisions before the lower-level ones) and has two communication phases. In the negotiation phase, agents determine the priority of decision-making by communicating hidden states of observations and comparing the value of intention, obtained by modeling the environment dynamics. In the launching phase, the upper-level agents take the lead in making decisions and then communicate their actions with the lower-level agents. Theoretically, we prove the policies learned by SeqComm are guaranteed to improve monotonically and converge. Empirically, we show that SeqComm outperforms existing methods in various cooperative multi-agent tasks.