BGC: Multi-Agent Group Belief with Graph Clustering

TL;DR

This paper introduces BGC, a semi-communication multi-agent reinforcement learning method that uses graph clustering to enable agents to develop shared beliefs and coordinate effectively without direct communication.

Contribution

It proposes a novel Belief in Graph Clustering (BGC) framework that incorporates group-based belief consensus and a hyper-network for improved multi-agent cooperation.

Findings

Significant performance improvement on SMAC benchmark.

Maintains high performance as the number of agents increases.

Effectively enables coordination without explicit communication.

Abstract

Recent advances have witnessed that value decomposed-based multi-agent reinforcement learning methods make an efficient performance in coordination tasks. Most current methods assume that agents can make communication to assist decisions, which is impractical in some situations. In this paper, we propose a semi-communication method to enable agents can exchange information without communication. Specifically, we introduce a group concept to help agents learning a belief which is a type of consensus. With this consensus, adjacent agents tend to accomplish similar sub-tasks to achieve cooperation. We design a novel agent structure named Belief in Graph Clustering(BGC), composed of an agent characteristic module, a belief module, and a fusion module. To represent each agent characteristic, we use an MLP-based characteristic module to generate agent unique features. Inspired by the neighborhood cognitive consistency, we propose a group-based module to divide adjacent agents into a small group and minimize in-group agents' beliefs to accomplish similar sub-tasks. Finally, we use a hyper-network to merge these features and produce agent actions. To overcome the agent consistent problem brought by GAT, a split loss is introduced to distinguish different agents. Results reveal that the proposed method achieves a significant improvement in the SMAC benchmark. Because of the group concept, our approach maintains excellent performance with an increase in the number of agents.