Emergence of Linguistic Communication from Referential Games with Symbolic and Pixel Input

TL;DR

This paper demonstrates that neural agents trained on referential communication games with raw pixel inputs can develop structured, compositional language, especially when the input data is inherently structured.

Contribution

It extends previous work by enabling agents to learn from raw pixel data, showing the impact of input structure on emergent communication protocols.

Findings

Structured input data promotes the emergence of compositional language.

Agents trained on pixel data can develop meaningful communication protocols.

Input data structure influences the complexity of emergent language.

Abstract

The ability of algorithms to evolve or learn (compositional) communication protocols has traditionally been studied in the language evolution literature through the use of emergent communication tasks. Here we scale up this research by using contemporary deep learning methods and by training reinforcement-learning neural network agents on referential communication games. We extend previous work, in which agents were trained in symbolic environments, by developing agents which are able to learn from raw pixel data, a more challenging and realistic input representation. We find that the degree of structure found in the input data affects the nature of the emerged protocols, and thereby corroborate the hypothesis that structured compositional language is most likely to emerge when agents perceive the world as being structured.