Similarity-based cooperative equilibrium

TL;DR

This paper introduces a realistic setting where agents observe only a similarity score rather than full transparency, enabling cooperative outcomes in social dilemmas like the Prisoner's Dilemma, supported by theoretical proof and experimental results.

Contribution

It shows that similarity-based observation suffices for cooperation, extending prior full transparency models to more practical partial transparency scenarios.

Findings

Similarity scores enable cooperation comparable to full transparency.

Simple ML methods can learn cooperative behavior in the new setting.

Theoretical proof confirms the sufficiency of similarity observation for cooperation.

Abstract

As machine learning agents act more autonomously in the world, they will increasingly interact with each other. Unfortunately, in many social dilemmas like the one-shot Prisoner's Dilemma, standard game theory predicts that ML agents will fail to cooperate with each other. Prior work has shown that one way to enable cooperative outcomes in the one-shot Prisoner's Dilemma is to make the agents mutually transparent to each other, i.e., to allow them to access one another's source code (Rubinstein 1998, Tennenholtz 2004) -- or weights in the case of ML agents. However, full transparency is often unrealistic, whereas partial transparency is commonplace. Moreover, it is challenging for agents to learn their way to cooperation in the full transparency setting. In this paper, we introduce a more realistic setting in which agents only observe a single number indicating how similar they are to each other. We prove that this allows for the same set of cooperative outcomes as the full transparency setting. We also demonstrate experimentally that cooperation can be learned using simple ML methods.