Path to Stochastic Stability: Comparative Analysis of Stochastic Learning Dynamics in Games

TL;DR

This paper develops a framework to compare different stochastic learning dynamics in games, focusing on their paths to equilibrium and short- to medium-term behaviors, beyond their shared steady states.

Contribution

It introduces a novel comparative analysis framework for stochastic learning rules with identical steady states, exemplified through Log-Linear and Metropolis Learning dynamics.

Findings

Distinct paths to equilibrium for LLL and ML

Upper bounds on hitting times to Nash equilibria

Hierarchical state space decomposition into cycles

Abstract

Stochastic stability is a popular solution concept for stochastic learning dynamics in games. However, a critical limitation of this solution concept is its inability to distinguish between different learning rules that lead to the same steady-state behavior. We address this limitation for the first time and develop a framework for the comparative analysis of stochastic learning dynamics with different update rules but same steady-state behavior. We present the framework in the context of two learning dynamics: Log-Linear Learning (LLL) and Metropolis Learning (ML). Although both of these dynamics have the same stochastically stable states, LLL and ML correspond to different behavioral models for decision making. Moreover, we demonstrate through an example setup of sensor coverage game that for each of these dynamics, the paths to stochastically stable states exhibit distinctive behaviors. Therefore, we propose multiple criteria to analyze and quantify the differences in the short and medium run behavior of stochastic learning dynamics. We derive and compare upper bounds on the expected hitting time to the set of Nash equilibria for both LLL and ML. For the medium to long-run behavior, we identify a set of tools from the theory of perturbed Markov chains that result in a hierarchical decomposition of the state space into collections of states called cycles. We compare LLL and ML based on the proposed criteria and develop invaluable insights into the comparative behavior of the two dynamics.