Multi-agent reinforcement learning in the all-or-nothing public goods game on networks

TL;DR

This paper investigates how multi-agent reinforcement learning influences trust and cooperation in public goods games on networks, revealing that network density affects contribution levels and trust dynamics.

Contribution

It introduces a theoretical analysis of long-term outcomes in multi-agent reinforcement learning for public goods games on networks, highlighting the impact of network structure on cooperation.

Findings

Long-term outcomes are either full contribution or defection.

Dense networks lead to lower contribution rates.

Metastable states depend on local network characteristics.

Abstract

We study interpersonal trust by means of the all-or-nothing public goods game between agents on a network. The agents are endowed with the simple yet adaptive learning rule, exponential moving average, by which they estimate the behavior of their neighbors in the network. Theoretically we show that in the long-time limit this multi-agent reinforcement learning process always eventually results in indefinite contribution to the public good or indefinite defection (no agent contributing to the public good). However, by simulation of the pre-limit behavior, we see that on complex network structures there may be mixed states in which the process seems to stabilize before actual convergence to states in which agent beliefs and actions are all the same. In these metastable states the local network characteristics can determine whether agents have high or low trust in their neighbors. More generally it is found that more dense networks result in lower rates of contribution to the public good. This has implications for how one can spread global contribution toward a public good by enabling smaller local interactions.