Accelerated regularized learning in finite N-person games

TL;DR

This paper introduces accelerated learning algorithms for finite N-person games that leverage momentum to achieve superlinear convergence to Nash equilibria, outperforming traditional methods across various feedback settings.

Contribution

The paper proposes the FTXL family of accelerated learning methods, extending Nesterov's acceleration to game-theoretic online learning with broad applicability.

Findings

FTXL converges locally to strict Nash equilibria at a superlinear rate.

Achieves exponential speed-up over standard regularized learning methods.

Maintains convergence speed across diverse feedback structures, including bandit settings.

Abstract

Motivated by the success of Nesterov's accelerated gradient algorithm for convex minimization problems, we examine whether it is possible to achieve similar performance gains in the context of online learning in games. To that end, we introduce a family of accelerated learning methods, which we call "follow the accelerated leader" (FTXL), and which incorporates the use of momentum within the general framework of regularized learning - and, in particular, the exponential/multiplicative weights algorithm and its variants. Drawing inspiration and techniques from the continuous-time analysis of Nesterov's algorithm, we show that FTXL converges locally to strict Nash equilibria at a superlinear rate, achieving in this way an exponential speed-up over vanilla regularized learning methods (which, by comparison, converge to strict equilibria at a geometric, linear rate). Importantly, FTXL maintains its superlinear convergence rate in a broad range of feedback structures, from deterministic, full information models to stochastic, realization-based ones, and even when run with bandit, payoff-based information, where players are only able to observe their individual realized payoffs.