Heterogeneous Beliefs and Multi-Population Learning in Network Games

TL;DR

This paper models multi-population learning with heterogeneous beliefs in network games, showing how beliefs evolve and influence equilibrium selection, with convergence to Quantal Response Equilibria and belief homogenization over time.

Contribution

It introduces a PDE-based model for multi-population learning with heterogeneous beliefs and analyzes equilibrium convergence and the role of initial belief heterogeneity.

Findings

Beliefs evolve according to PDEs similar to transport phenomena.

SFP converges to Quantal Response Equilibria in various network games.

Beliefs homogenize over time, affecting equilibrium selection in coordination games.

Abstract

The effect of population heterogeneity in multi-agent learning is practically relevant but remains far from being well-understood. Motivated by this, we introduce a model of multi-population learning that allows for heterogeneous beliefs within each population and where agents respond to their beliefs via smooth fictitious play (SFP).We show that the system state -- a probability distribution over beliefs -- evolves according to a system of partial differential equations akin to the continuity equations that commonly desccribe transport phenomena in physical systems. We establish the convergence of SFP to Quantal Response Equilibria in different classes of games capturing both network competition as well as network coordination. We also prove that the beliefs will eventually homogenize in all network games. Although the initial belief heterogeneity disappears in the limit, we show that it plays a crucial role for equilibrium selection in the case of coordination games as it helps select highly desirable equilibria. Contrary, in the case of network competition, the resulting limit behavior is independent of the initialization of beliefs, even when the underlying game has many distinct Nash equilibria.