Think How Your Teammates Think: Active Inference Can Benefit Decentralized Execution

TL;DR

This paper introduces a non-communication multi-agent reinforcement learning framework that models teammates' decision processes through active inference, improving coordination without relying on communication.

Contribution

It proposes a novel local observation-based modeling approach that constructs teammate cognition via perception, belief, and action portraits, enhancing decentralized cooperation.

Findings

Outperforms existing methods on SMAC, SMACv2, MPE, and GRF benchmarks.

Effectively models teammates' decision logic without communication.

Improves coordination and collaboration in multi-agent systems.

Abstract

In multi-agent systems, explicit cognition of teammates' decision logic serves as a critical factor in facilitating coordination. Communication (i.e., ``\textit{Tell}'') can assist in the cognitive development process by information dissemination, yet it is inevitably subject to real-world constraints such as noise, latency, and attacks. Therefore, building the understanding of teammates' decisions without communication remains challenging. To address this, we propose a novel non-communication MARL framework that realizes the construction of cognition through local observation-based modeling (i.e., \textit{``Think''}). Our framework enables agents to model teammates' \textbf{active inference} process. At first, the proposed method produces three teammate portraits: perception-belief-action. Specifically, we model the teammate's decision process as follows: 1) Perception: observing environments; 2) Belief: forming beliefs; 3) Action: making decisions. Then, we selectively integrate the belief portrait into the decision process based on the accuracy and relevance of the perception portrait. This enables the selection of cooperative teammates and facilitates effective collaboration. Extensive experiments on the SMAC, SMACv2, MPE, and GRF benchmarks demonstrate the superior performance of our method.