Learning Emergent Discrete Message Communication for Cooperative Reinforcement Learning

TL;DR

This paper introduces a method for multi-agent reinforcement learning where agents learn to communicate using discrete, human-interpretable messages, achieving comparable performance to continuous communication with enhanced interpretability.

Contribution

The paper proposes a novel approach for learning discrete message protocols in MARL using self-attention, improving interpretability without sacrificing performance.

Findings

Discrete messages achieve similar performance to continuous messages

Smaller vocabulary size enhances interpretability

Humans can interactively send messages to agents

Abstract

Communication is a important factor that enables agents work cooperatively in multi-agent reinforcement learning (MARL). Most previous work uses continuous message communication whose high representational capacity comes at the expense of interpretability. Allowing agents to learn their own discrete message communication protocol emerged from a variety of domains can increase the interpretability for human designers and other agents.This paper proposes a method to generate discrete messages analogous to human languages, and achieve communication by a broadcast-and-listen mechanism based on self-attention. We show that discrete message communication has performance comparable to continuous message communication but with much a much smaller vocabulary size.Furthermore, we propose an approach that allows humans to interactively send discrete messages to agents.