Speaking Your Language: Spatial Relationships in Interpretable Emergent Communication

TL;DR

This paper shows that agents can develop an interpretable language to describe spatial relationships in observations, achieving high accuracy in referential tasks and human interpretability.

Contribution

It introduces the emergence of spatial relationship references in agent communication, a novel aspect not previously demonstrated in emergent language research.

Findings

Agents achieved over 90% accuracy in spatial reference tasks.

Emergent language is interpretable by humans with over 78% translation accuracy.

Agents use a mix of compositional and non-compositional messages.

Abstract

Effective communication requires the ability to refer to specific parts of an observation in relation to others. While emergent communication literature shows success in developing various language properties, no research has shown the emergence of such positional references. This paper demonstrates how agents can communicate about spatial relationships within their observations. The results indicate that agents can develop a language capable of expressing the relationships between parts of their observation, achieving over 90% accuracy when trained in a referential game which requires such communication. Using a collocation measure, we demonstrate how the agents create such references. This analysis suggests that agents use a mixture of non-compositional and compositional messages to convey spatial relationships. We also show that the emergent language is interpretable by humans. The translation accuracy is tested by communicating with the receiver agent, where the receiver achieves over 78% accuracy using parts of this lexicon, confirming that the interpretation of the emergent language was successful.