Mixed Cooperative-Competitive Communication Using Multi-Agent Reinforcement Learning

TL;DR

This paper explores multi-agent reinforcement learning with communication in mixed cooperative-competitive environments, analyzing how communication privacy affects team performance and demonstrating that shared communication can reduce effectiveness.

Contribution

It applies differentiable inter-agent learning to mixed settings and compares private versus overheard communication, revealing impacts on team performance.

Findings

Shared communication decreases team performance compared to private communication.

Communicating agents can achieve performance similar to fully observable agents.

Overhearing communication negatively impacts the communicating team's results.

Abstract

By using communication between multiple agents in multi-agent environments, one can reduce the effects of partial observability by combining one agent's observation with that of others in the same dynamic environment. While a lot of successful research has been done towards communication learning in cooperative settings, communication learning in mixed cooperative-competitive settings is also important and brings its own complexities such as the opposing team overhearing the communication. In this paper, we apply differentiable inter-agent learning (DIAL), designed for cooperative settings, to a mixed cooperative-competitive setting. We look at the difference in performance between communication that is private for a team and communication that can be overheard by the other team. Our research shows that communicating agents are able to achieve similar performance to fully observable agents after a given training period in our chosen environment. Overall, we find that sharing communication across teams results in decreased performance for the communicating team in comparison to results achieved with private communication.