Multi-Agent Synchronization Tasks

TL;DR

This paper introduces Multi-Agent Synchronization Tasks (MSTs), a new benchmark for evaluating coordination in multi-agent reinforcement learning, revealing current algorithms struggle with scalability beyond two agents in communication-dependent scenarios.

Contribution

The paper defines MSTs as a new class of multi-agent tasks and provides a detailed example, Synchronized Predator-Prey, to evaluate state-of-the-art MARL algorithms' coordination capabilities.

Findings

Current MARL algorithms fail to scale beyond 2-agent coordination in MSTs.

Existing algorithms show limitations in handling communication-dependent tasks.

Results question the effectiveness of recent SOTA methods for complex multi-agent coordination.

Abstract

In multi-agent reinforcement learning (MARL), coordination plays a crucial role in enhancing agents' performance beyond what they could achieve through cooperation alone. The interdependence of agents' actions, coupled with the need for communication, leads to a domain where effective coordination is crucial. In this paper, we introduce and define $\textit{Multi-Agent Synchronization Tasks}$ (MSTs), a novel subset of multi-agent tasks. We describe one MST, that we call $\textit{Synchronized Predator-Prey}$, offering a detailed description that will serve as the basis for evaluating a selection of recent state-of-the-art (SOTA) MARL algorithms explicitly designed to address coordination challenges through the use of communication strategies. Furthermore, we present empirical evidence that reveals the limitations of the algorithms assessed to solve MSTs, demonstrating their inability to scale effectively beyond 2-agent coordination tasks in scenarios where communication is a requisite component. Finally, the results raise questions about the applicability of recent SOTA approaches for complex coordination tasks (i.e. MSTs) and prompt further exploration into the underlying causes of their limitations in this context.