Multi-Agent Intention Sharing via Leader-Follower Forest

TL;DR

This paper introduces the leader-follower forest (LFF), a hierarchical structure for intention sharing in multi-agent reinforcement learning that reduces message deceiving and improves coordination in partially observable environments.

Contribution

The paper proposes the LFF model for hierarchical intention sharing and a two-stage training algorithm, addressing message deceiving in multi-agent communication.

Findings

LFF effectively eliminates message deceiving.

IS-LFF outperforms state-of-the-art algorithms.

Improves coordination in partially observable MARL environments.

Abstract

Intention sharing is crucial for efficient cooperation under partially observable environments in multi-agent reinforcement learning (MARL). However, message deceiving, i.e., a mismatch between the propagated intentions and the final decisions, may happen when agents change strategies simultaneously according to received intentions. Message deceiving leads to potential miscoordination and difficulty for policy learning. This paper proposes the leader-follower forest (LFF) to learn the hierarchical relationship between agents based on interdependencies, achieving one-sided intention sharing in multi-agent communication. By limiting the flowings of intentions through directed edges, intention sharing via LFF (IS-LFF) can eliminate message deceiving effectively and achieve better coordination. In addition, a twostage learning algorithm is proposed to train the forest and the agent network. We evaluate IS-LFF on multiple partially observable MARL benchmarks, and the experimental results show that our method outperforms state-of-the-art communication algorithms.