Emergent cooperation through mutual information maximization

TL;DR

This paper introduces a decentralized deep reinforcement learning method that encourages cooperation among agents by maximizing mutual information, leading to emergent cooperative behavior in social dilemmas.

Contribution

It proposes a novel auxiliary objective of mutual information maximization to promote cooperation in multi-agent reinforcement learning systems.

Findings

Mutual information maximization enhances cooperation among agents.

The proposed method outperforms systems without the auxiliary objective.

Cooperative behavior emerges in social dilemmas through this approach.

Abstract

With artificial intelligence systems becoming ubiquitous in our society, its designers will soon have to start to consider its social dimension, as many of these systems will have to interact among them to work efficiently. With this in mind, we propose a decentralized deep reinforcement learning algorithm for the design of cooperative multi-agent systems. The algorithm is based on the hypothesis that highly correlated actions are a feature of cooperative systems, and hence, we propose the insertion of an auxiliary objective of maximization of the mutual information between the actions of agents in the learning problem. Our system is applied to a social dilemma, a problem whose optimal solution requires that agents cooperate to maximize a macroscopic performance function despite the divergent individual objectives of each agent. By comparing the performance of the proposed system to a system without the auxiliary objective, we conclude that the maximization of mutual information among agents promotes the emergence of cooperation in social dilemmas.