Regret Minimization in Population Network Games: Vanishing Heterogeneity and Convergence to Equilibria

TL;DR

This paper studies how heterogeneity among agents in large-scale multi-agent systems diminishes over time through regret-matching, leading to consensus and convergence to equilibria, thus advancing understanding of multi-agent learning dynamics.

Contribution

It introduces a novel analysis of regret distribution dynamics, showing heterogeneity vanishes and agents converge to equilibria in diverse multi-agent settings.

Findings

Variance of regret distribution decreases over time

Agents reach consensus despite initial heterogeneity

Convergence to quantal response equilibria proven in various settings

Abstract

Understanding and predicting the behavior of large-scale multi-agents in games remains a fundamental challenge in multi-agent systems. This paper examines the role of heterogeneity in equilibrium formation by analyzing how smooth regret-matching drives a large number of heterogeneous agents with diverse initial policies toward unified behavior. By modeling the system state as a probability distribution of regrets and analyzing its evolution through the continuity equation, we uncover a key phenomenon in diverse multi-agent settings: the variance of the regret distribution diminishes over time, leading to the disappearance of heterogeneity and the emergence of consensus among agents. This universal result enables us to prove convergence to quantal response equilibria in both competitive and cooperative multi-agent settings. Our work advances the theoretical understanding of multi-agent learning and offers a novel perspective on equilibrium selection in diverse game-theoretic scenarios.