On the Pitfalls of Measuring Emergent Communication

TL;DR

This paper critically examines existing metrics for measuring emergent communication in multi-agent systems, revealing potential misleading indicators and proposing more nuanced evaluation tools for complex environments.

Contribution

It highlights the limitations of current metrics, demonstrates a scenario where communication appears to emerge without functional impact, and offers recommendations for better measurement practices.

Findings

Existing metrics can be misleading about true communication

Agents can appear to communicate without influencing the environment

Recommendations for more accurate measurement of emergent communication

Abstract

How do we know if communication is emerging in a multi-agent system? The vast majority of recent papers on emergent communication show that adding a communication channel leads to an increase in reward or task success. This is a useful indicator, but provides only a coarse measure of the agent's learned communication abilities. As we move towards more complex environments, it becomes imperative to have a set of finer tools that allow qualitative and quantitative insights into the emergence of communication. This may be especially useful to allow humans to monitor agents' behaviour, whether for fault detection, assessing performance, or even building trust. In this paper, we examine a few intuitive existing metrics for measuring communication, and show that they can be misleading. Specifically, by training deep reinforcement learning agents to play simple matrix games augmented with a communication channel, we find a scenario where agents appear to communicate (their messages provide information about their subsequent action), and yet the messages do not impact the environment or other agent in any way. We explain this phenomenon using ablation studies and by visualizing the representations of the learned policies. We also survey some commonly used metrics for measuring emergent communication, and provide recommendations as to when these metrics should be used.