Optimization, Learning, and Games with Predictable Sequences

TL;DR

This paper explores the use of Optimistic Mirror Descent in online learning, optimization, and game theory, demonstrating new algorithms and convergence results for saddle-point problems, zero-sum games, and convex programming.

Contribution

It introduces novel applications of Optimistic Mirror Descent, including extensions to Holder-smooth functions, convergence to equilibrium in zero-sum games, and algorithms for convex optimization problems.

Findings

Mirror Prox recovered for offline optimization

Convergence to minimax equilibrium in zero-sum games at rate O((log T)/T)

New algorithm for approximate Max Flow problem

Abstract

We provide several applications of Optimistic Mirror Descent, an online learning algorithm based on the idea of predictable sequences. First, we recover the Mirror Prox algorithm for offline optimization, prove an extension to Holder-smooth functions, and apply the results to saddle-point type problems. Next, we prove that a version of Optimistic Mirror Descent (which has a close relation to the Exponential Weights algorithm) can be used by two strongly-uncoupled players in a finite zero-sum matrix game to converge to the minimax equilibrium at the rate of O((log T)/T). This addresses a question of Daskalakis et al 2011. Further, we consider a partial information version of the problem. We then apply the results to convex programming and exhibit a simple algorithm for the approximate Max Flow problem.