Mean Field Asymptotics of Markov Decision Evolutionary Games and Teams

TL;DR

This paper develops a rigorous framework for analyzing large population Markov games using mean field asymptotics, deriving limiting differential equations, and constructing near-optimal strategies for finite populations.

Contribution

It introduces a new mean field approach for Markov decision evolutionary games, characterizes the limit behavior, and links microscopic models to macroscopic mean field games.

Findings

Weak convergence of individual-player processes to jump processes

Characterization of limit solutions for team and game problems

Construction of near-optimal strategies for large but finite populations

Abstract

We introduce Mean Field Markov games with $N$ players, in which each individual in a large population interacts with other randomly selected players. The states and actions of each player in an interaction together determine the instantaneous payoff for all involved players. They also determine the transition probabilities to move to the next state. Each individual wishes to maximize the total expected discounted payoff over an infinite horizon. We provide a rigorous derivation of the asymptotic behavior of this system as the size of the population grows to infinity. Under indistinguishability per type assumption, we show that under any Markov strategy, the random process consisting of one specific player and the remaining population converges weakly to a jump process driven by the solution of a system of differential equations. We characterize the solutions to the team and to the game problems at the limit of infinite population and use these to construct near optimal strategies for the case of a finite, but large, number of players. We show that the large population asymptotic of the microscopic model is equivalent to a (macroscopic) mean field stochastic game in which a local interaction is described by a single player against a population profile (the mean field limit). We illustrate our model to derive the equations for a dynamic evolutionary Hawk and Dove game with energy level.