Visual Theory of Mind Enables the Invention of Proto-Writing

TL;DR

This paper introduces a multi-agent reinforcement learning framework that models how visual theory of mind can enable the emergence of proto-writing through pictographic communication, shedding light on early cognitive and cultural processes.

Contribution

It presents a novel Signification Game testbed and a model of inferential communication that incorporates visual theory of mind, advancing understanding of proto-writing origins.

Findings

Agents successfully use pictographs to communicate actions.

The model demonstrates the importance of visual theory of mind in emergent proto-writing.

Provides insights into cognitive processes behind early symbolic systems.

Abstract

Symbolic writing systems are graphical semiotic codes that are ubiquitous in modern society but are otherwise absent in the animal kingdom. Anthropological evidence suggests that the earliest forms of some writing systems originally consisted of iconic pictographs, which signify their referent via visual resemblance. While previous studies have examined the emergence and, separately, the evolution of pictographic systems through a computational lens, most employ non-naturalistic methodologies that make it difficult to draw clear analogies to human and animal cognition. We develop a multi-agent reinforcement learning testbed for emergent communication called a Signification Game, and formulate a model of inferential communication that enables agents to leverage visual theory of mind to communicate actions using pictographs. Our model, which is situated within a broader formalism for animal communication, sheds light on the cognitive and cultural processes underlying the emergence of proto-writing.