Emergence of Communication in an Interactive World with Consistent Speakers

TL;DR

This paper introduces a new environment and algorithm for training agents to develop communication from raw pixels, improving stability and context-independence over traditional policy gradient methods.

Contribution

It presents a novel environment and a representation-based training algorithm that enhances the emergence of consistent, context-independent communication in agents.

Findings

Our algorithm outperforms policy gradient in stability and performance.

The method increases context-independence in emergent communication.

Empirical results demonstrate improved communication consistency.

Abstract

Training agents to communicate with one another given task-based supervision only has attracted considerable attention recently, due to the growing interest in developing models for human-agent interaction. Prior work on the topic focused on simple environments, where training using policy gradient was feasible despite the non-stationarity of the agents during training. In this paper, we present a more challenging environment for testing the emergence of communication from raw pixels, where training using policy gradient fails. We propose a new model and training algorithm, that utilizes the structure of a learned representation space to produce more consistent speakers at the initial phases of training, which stabilizes learning. We empirically show that our algorithm substantially improves performance compared to policy gradient. We also propose a new alignment-based metric for measuring context-independence in emerged communication and find our method increases context-independence compared to policy gradient and other competitive baselines.