Multi-Agent Cooperation via Unsupervised Learning of Joint Intentions

TL;DR

This paper introduces a novel multi-agent reinforcement learning approach that learns joint intentions in an unsupervised manner, improving cooperation and performance in complex multi-agent tasks.

Contribution

It proposes a hierarchical MARL framework that autonomously learns joint intentions in a latent space, addressing non-monotonic return issues and enhancing interpretability.

Findings

Significant performance improvements in StarCraft benchmarks

Effective learning of meaningful joint intentions

Adaptability to different agent configurations

Abstract

The field of cooperative multi-agent reinforcement learning (MARL) has seen widespread use in addressing complex coordination tasks. While value decomposition methods in MARL have been popular, they have limitations in solving tasks with non-monotonic returns, restricting their general application. Our work highlights the significance of joint intentions in cooperation, which can overcome non-monotonic problems and increase the interpretability of the learning process. To this end, we present a novel MARL method that leverages learnable joint intentions. Our method employs a hierarchical framework consisting of a joint intention policy and a behavior policy to formulate the optimal cooperative policy. The joint intentions are autonomously learned in a latent space through unsupervised learning and enable the method adaptable to different agent configurations. Our results demonstrate significant performance improvements in both the StarCraft micromanagement benchmark and challenging MAgent domains, showcasing the effectiveness of our method in learning meaningful joint intentions.