The emergence of numerical representations in communicating artificial agents

TL;DR

This study investigates whether communication pressure alone can lead artificial agents to develop numerical representations similar to human numerals, finding that while accuracy improves, compositionality and generalization do not emerge naturally.

Contribution

The paper demonstrates that communication pressure alone results in high-accuracy numerosity communication but does not produce compositional or generalizable numerical codes in neural agents.

Findings

Agents achieve high in-distribution accuracy in both symbolic and iconic channels.

Emergent codes are non-compositional and fail to generalize to unseen numerosities.

Additional pressures are necessary for compositionality and generalization.

Abstract

Human languages provide efficient systems for expressing numerosities, but whether the sheer pressure to communicate is enough for numerical representations to arise in artificial agents, and whether the emergent codes resemble human numerals at all, remains an open question. We study two neural network-based agents that must communicate numerosities in a referential game using either discrete tokens or continuous sketches, thus exploring both symbolic and iconic representations. Without any pre-defined numeric concepts, the agents achieve high in-distribution communication accuracy in both communication channels and converge on high-precision symbol-meaning mappings. However, the emergent code is non-compositional: the agents fail to derive systematic messages for unseen numerosities, typically reusing the symbol of the highest trained numerosity (discrete), or collapsing extrapolated values onto a single sketch (continuous). We conclude that the communication pressure alone suffices for precise transmission of learned numerosities, but additional pressures are needed to yield compositional codes and generalisation abilities.