Defending Compositionality in Emergent Languages

TL;DR

This paper investigates the role of compositionality in emergent languages, demonstrating its importance for generalization in neural communication systems through a two-agent game analysis.

Contribution

It clarifies the significance of compositionality for generalization, countering recent claims that neural networks can generalize without it.

Findings

Compositionality is crucial for successful generalization.

Neural networks benefit from compositional structures in language.

Proper dataset evaluation reveals the importance of compositionality.

Abstract

Compositionality has traditionally been understood as a major factor in productivity of language and, more broadly, human cognition. Yet, recently, some research started to question its status, showing that artificial neural networks are good at generalization even without noticeable compositional behavior. We argue that some of these conclusions are too strong and/or incomplete. In the context of a two-agent communication game, we show that compositionality indeed seems essential for successful generalization when the evaluation is done on a proper dataset.