Emergent Communication in a Multi-Modal, Multi-Step Referential Game
Katrina Evtimova, Andrew Drozdov, Douwe Kiela, Kyunghyun Cho

TL;DR
This paper introduces a multi-modal, multi-step referential game where agents develop a natural language-like communication protocol to identify objects, demonstrating improved accuracy and generalization with increased communication bandwidth.
Contribution
It presents a novel multi-modal, multi-step referential game framework that enables agents to develop more natural and flexible communication protocols.
Findings
Emergent communication closely resembles natural language.
Increased communication bandwidth enhances generalization.
Gradual information exchange improves prediction accuracy.
Abstract
Inspired by previous work on emergent communication in referential games, we propose a novel multi-modal, multi-step referential game, where the sender and receiver have access to distinct modalities of an object, and their information exchange is bidirectional and of arbitrary duration. The multi-modal multi-step setting allows agents to develop an internal communication significantly closer to natural language, in that they share a single set of messages, and that the length of the conversation may vary according to the difficulty of the task. We examine these properties empirically using a dataset consisting of images and textual descriptions of mammals, where the agents are tasked with identifying the correct object. Our experiments indicate that a robust and efficient communication protocol emerges, where gradual information exchange informs better predictions and higher…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsLanguage and cultural evolution · Evolutionary Game Theory and Cooperation · Artificial Intelligence in Games
