Empirical Centroid Fictitious Play: An Approach For Distributed Learning In Multi-Agent Games

TL;DR

This paper introduces an efficient distributed learning algorithm called empirical centroid fictitious play (ECFP) for large-scale multi-agent games, proving its convergence to Nash equilibria in potential games with minimal communication.

Contribution

The paper proposes ECFP, a scalable variant of fictitious play that responds to the action centroid, with convergence guarantees and a distributed implementation for large multi-agent systems.

Findings

ECFP converges to Nash equilibria in potential games.

Distributed ECFP achieves consensus on equilibrium with sparse communication.

The algorithm reduces computational and communication complexity in large-scale settings.

Abstract

The paper is concerned with distributed learning in large-scale games. The well-known fictitious play (FP) algorithm is addressed, which, despite theoretical convergence results, might be impractical to implement in large-scale settings due to intense computation and communication requirements. An adaptation of the FP algorithm, designated as the empirical centroid fictitious play (ECFP), is presented. In ECFP players respond to the centroid of all players' actions rather than track and respond to the individual actions of every player. Convergence of the ECFP algorithm in terms of average empirical frequency (a notion made precise in the paper) to a subset of the Nash equilibria is proven under the assumption that the game is a potential game with permutation invariant potential function. A more general formulation of ECFP is then given (which subsumes FP as a special case) and convergence results are given for the class of potential games. Furthermore, a distributed formulation of the ECFP algorithm is presented, in which, players endowed with a (possibly sparse) preassigned communication graph, engage in local, non-strategic information exchange to eventually agree on a common equilibrium. Convergence results are proven for the distributed ECFP algorithm.