Stochastic Approximation, Cooperative Dynamics and Supermodular Games

TL;DR

This paper analyzes a stochastic approximation algorithm, showing non-convergence to certain sets and establishing convergence in cooperative supermodular games, extending previous results on stochastic fictitious play.

Contribution

It proves non-convergence to repulsive sets and extends convergence results for stochastic fictitious play in supermodular games under mild assumptions.

Findings

Non-convergence to repulsive sets for the stochastic process.

Convergence of the process in cooperative supermodular games.

Extension of previous convergence results in game dynamics.

Abstract

This paper considers a stochastic approximation algorithm, with decreasing step size and martingale difference noise. Under very mild assumptions, we prove the non convergence of this process toward a certain class of repulsive sets for the associated ordinary differential equation (ODE). We then use this result to derive the convergence of the process when the ODE is cooperative in the sense of [Hirsch, 1985]. In particular, this allows us to extend significantly the main result of [Hofbauer and Sandholm, 2002] on the convergence of stochastic fictitious play in supermodular games.