Learning Equilibrium with Estimated Payoffs in Population Games

TL;DR

This paper analyzes how payoff estimation errors influence the convergence to equilibrium in population games and proposes a dynamic revision rate to ensure convergence despite these errors.

Contribution

It introduces a novel analysis of payoff estimation errors in population games and designs a time-varying strategy revision rate to guarantee convergence.

Findings

Estimation errors can hinder convergence to equilibrium.

A time-varying revision rate improves convergence robustness.

Simulation confirms effectiveness of the proposed method.

Abstract

We study a multi-agent decision problem in population games, where agents select from multiple available strategies and continually revise their selections based on the payoffs associated with these strategies. Unlike conventional population game formulations, we consider a scenario where agents must estimate the payoffs through local measurements and communication with their neighbors. By employing task allocation games -- dynamic extensions of conventional population games -- we examine how errors in payoff estimation by individual agents affect the convergence of the strategy revision process. Our main contribution is an analysis of how estimation errors impact the convergence of the agents' strategy profile to equilibrium. Based on the analytical results, we propose a design for a time-varying strategy revision rate to guarantee convergence. Simulation studies illustrate how the proposed method for updating the revision rate facilitates convergence to equilibrium.