Fast Convergence of Regularized Learning in Games

TL;DR

This paper demonstrates that certain regularized learning algorithms with recency bias enable faster convergence to equilibrium and efficiency in multiplayer games, improving upon previous worst-case rates.

Contribution

It introduces a class of regularized algorithms with recency bias that achieve faster convergence rates in multiplayer games, extending prior two-player zero-sum game results.

Findings

Regret decays at O(T^{-3/4}) for individual players.

Sum of utilities converges at O(T^{-1}) to an approximate optimum.

Black-box reduction achieves (T^{-1/2}) rates against adversaries.

Abstract

We show that natural classes of regularized learning algorithms with a form of recency bias achieve faster convergence rates to approximate efficiency and to coarse correlated equilibria in multiplayer normal form games. When each player in a game uses an algorithm from our class, their individual regret decays at $O(T^{-3/4})$, while the sum of utilities converges to an approximate optimum at $O(T^{-1})$--an improvement upon the worst case $O(T^{-1/2})$ rates. We show a black-box reduction for any algorithm in the class to achieve $\tilde{O}(T^{-1/2})$ rates against an adversary, while maintaining the faster rates against algorithms in the class. Our results extend those of [Rakhlin and Shridharan 2013] and [Daskalakis et al. 2014], who only analyzed two-player zero-sum games for specific algorithms.