Improved Cooperation by Exploiting a Common Signal

TL;DR

This paper demonstrates that introducing a common periodic signal among artificial agents enhances the emergence of sustainable cooperation in large-scale, low-observability common-pool resource environments, inspired by human conventions.

Contribution

It shows that a shared signal can facilitate the development of temporal conventions, significantly improving cooperation and sustainability in decentralized multi-agent systems.

Findings

Social welfare increased by up to 3306% with the signal.

Sustainability range expanded by up to 300%.

Convergence speed improved by up to 53%.

Abstract

Can artificial agents benefit from human conventions? Human societies manage to successfully self-organize and resolve the tragedy of the commons in common-pool resources, in spite of the bleak prediction of non-cooperative game theory. On top of that, real-world problems are inherently large-scale and of low observability. One key concept that facilitates human coordination in such settings is the use of conventions. Inspired by human behavior, we investigate the learning dynamics and emergence of temporal conventions, focusing on common-pool resources. Extra emphasis was given in designing a realistic evaluation setting: (a) environment dynamics are modeled on real-world fisheries, (b) we assume decentralized learning, where agents can observe only their own history, and (c) we run large-scale simulations (up to 64 agents). Uncoupled policies and low observability make cooperation hard to achieve; as the number of agents grow, the probability of taking a correct gradient direction decreases exponentially. By introducing an arbitrary common signal (e.g., date, time, or any periodic set of numbers) as a means to couple the learning process, we show that temporal conventions can emerge and agents reach sustainable harvesting strategies. The introduction of the signal consistently improves the social welfare (by 258% on average, up to 3306%), the range of environmental parameters where sustainability can be achieved (by 46% on average, up to 300%), and the convergence speed in low abundance settings (by 13% on average, up to 53%).