Second-Order Mirror Descent: Convergence in Games Beyond Averaging and Discounting

TL;DR

This paper introduces a second-order mirror descent (MD2) dynamics for continuous-time game convergence, achieving stability without averaging or discounting, and extends to stochastic discrete-time settings with noisy data.

Contribution

The paper proposes MD2, a novel second-order mirror descent method that ensures convergence to variationally stable states without traditional averaging or discounting techniques.

Findings

MD2 converges to variationally stable states in continuous time.

MD2 achieves exponential convergence to strong VSS with slight modifications.

Discrete-time MD2 converges under noisy observations using stochastic approximation.

Abstract

In this paper, we propose a second-order extension of the continuous-time game-theoretic mirror descent (MD) dynamics, referred to as MD2, which provably converges to mere (but not necessarily strict) variationally stable states (VSS) without using common auxiliary techniques such as time-averaging or discounting. We show that MD2 enjoys no-regret as well as an exponential rate of convergence towards strong VSS upon a slight modification. MD2 can also be used to derive many novel continuous-time primal-space dynamics. We then use stochastic approximation techniques to provide a convergence guarantee of discrete-time MD2 with noisy observations towards interior mere VSS. Selected simulations are provided to illustrate our results.