ReferentialGym: A Nomenclature and Framework for Language Emergence & Grounding in (Visual) Referential Games

TL;DR

This paper introduces a standardized terminology and a comprehensive deep learning framework, ReferentialGym, to facilitate research on language emergence and grounding in AI, aiming to improve human-machine communication.

Contribution

It proposes a clear nomenclature for the field and provides an open-source PyTorch framework with baseline algorithms to support future research.

Findings

Baseline implementations of key algorithms provided

Framework supports diverse approaches and metrics

Aims to standardize and accelerate research in language emergence

Abstract

Natural languages are powerful tools wielded by human beings to communicate information and co-operate towards common goals. Their values lie in some main properties like compositionality, hierarchy and recurrent syntax, which computational linguists have been researching the emergence of in artificial languages induced by language games. Only relatively recently, the AI community has started to investigate language emergence and grounding working towards better human-machine interfaces. For instance, interactive/conversational AI assistants that are able to relate their vision to the ongoing conversation. This paper provides two contributions to this research field. Firstly, a nomenclature is proposed to understand the main initiatives in studying language emergence and grounding, accounting for the variations in assumptions and constraints. Secondly, a PyTorch based deep learning framework is introduced, entitled ReferentialGym, which is dedicated to furthering the exploration of language emergence and grounding. By providing baseline implementations of major algorithms and metrics, in addition to many different features and approaches, ReferentialGym attempts to ease the entry barrier to the field and provide the community with common implementations.