Intrinsically Motivated Compositional Language Emergence

TL;DR

This paper introduces an intrinsic reward framework in reinforcement learning to enhance the compositionality of emergent communication between agents, outperforming previous methods constrained by limited channel capacity.

Contribution

It demonstrates that intrinsic rewards, rather than channel capacity limits, significantly improve the compositionality of emergent languages in multi-agent communication.

Findings

Intrinsic rewards increase compositionality scores by approximately 1.5-2 times.

The framework is effective across three different referential game setups, including a novel environment gComm.

Limited channel capacity alone is insufficient for achieving high compositionality.

Abstract

Recently, there has been a great deal of research in emergent communication on artificial agents interacting in simulated environments. Recent studies have revealed that, in general, emergent languages do not follow the compositionality patterns of natural language. To deal with this, existing works have proposed a limited channel capacity as an important constraint for learning highly compositional languages. In this paper, we show that this is not a sufficient condition and propose an intrinsic reward framework for improving compositionality in emergent communication. We use a reinforcement learning setting with two agents -- a \textit{task-aware} Speaker and a \textit{state-aware} Listener that are required to communicate to perform a set of tasks. Through our experiments on three different referential game setups, including a novel environment gComm, we show intrinsic rewards improve compositionality scores by $\approx \mathbf{1.5-2}$ times that of existing frameworks that use limited channel capacity.