Heterogeneous Multi-agent Zero-Shot Coordination by Coevolution

TL;DR

This paper introduces a coevolution-based method to enable agents to achieve zero-shot coordination with unseen partners in heterogeneous multi-agent tasks, addressing a gap in existing homogeneous-focused approaches.

Contribution

The paper pioneers the study of heterogeneous zero-shot coordination and proposes a coevolution framework that evolves agents and partners simultaneously for diverse tasks.

Findings

Effective in heterogeneous tasks

Outperforms existing methods in ZSC scenarios

Highlights importance of heterogeneity in training

Abstract

Generating agents that can achieve zero-shot coordination (ZSC) with unseen partners is a new challenge in cooperative multi-agent reinforcement learning (MARL). Recently, some studies have made progress in ZSC by exposing the agents to diverse partners during the training process. They usually involve self-play when training the partners, implicitly assuming that the tasks are homogeneous. However, many real-world tasks are heterogeneous, and hence previous methods may be inefficient. In this paper, we study the heterogeneous ZSC problem for the first time and propose a general method based on coevolution, which coevolves two populations of agents and partners through three sub-processes: pairing, updating and selection. Experimental results on various heterogeneous tasks highlight the necessity of considering the heterogeneous setting and demonstrate that our proposed method is a promising solution for heterogeneous ZSC tasks.