Learning Multiagent Communication with Backpropagation

TL;DR

This paper introduces CommNet, a neural model enabling multiple agents to learn continuous communication protocols during training, improving cooperative task performance and revealing interpretable strategies.

Contribution

It presents a neural communication model that learns communication protocols end-to-end, advancing multiagent cooperation without manual protocol design.

Findings

Agents learn effective communication strategies.

Improved performance over non-communicative baselines.

Some learned communication is interpretable.

Abstract

Many tasks in AI require the collaboration of multiple agents. Typically, the communication protocol between agents is manually specified and not altered during training. In this paper we explore a simple neural model, called CommNet, that uses continuous communication for fully cooperative tasks. The model consists of multiple agents and the communication between them is learned alongside their policy. We apply this model to a diverse set of tasks, demonstrating the ability of the agents to learn to communicate amongst themselves, yielding improved performance over non-communicative agents and baselines. In some cases, it is possible to interpret the language devised by the agents, revealing simple but effective strategies for solving the task at hand.