Self-organization in a distributed coordination game through heuristic rules

TL;DR

This paper explores how a large group of agents can self-organize to select a single equilibrium in a distributed coordination game using heuristic reinforcement learning rules, balancing speed and stability.

Contribution

It introduces heuristic rules based on reinforcement learning for equilibrium selection in a multi-agent coordination game, analyzing their effects on self-organization and stability.

Findings

Agents form clusters with different intensities based on heuristics

Most clusters are transient except one, indicating a dominant equilibrium

There is a trade-off between convergence speed and solution stability

Abstract

In this paper we consider a distributed coordination game played by a large number of agents with finite information sets, which characterizes emergence of a single dominant attribute out of a large number of competitors. Formally, $N$ agents play a coordination game repeatedly which has exactly $N$ Nash equilibria and all of the equilibria are equally preferred by the agents. The problem is to select one equilibrium out of $N$ possible equilibria in the least number of attempts. We propose a number of heuristic rules based on reinforcement learning to solve the coordination problem. We see that the agents self-organize into clusters with varying intensities depending on the heuristic rule applied although all clusters but one are transitory in most cases. Finally, we characterize a trade-off in terms of the time requirement to achieve a degree of stability in strategies and the efficiency of such a solution.