Extracting Dual Solutions via Primal Optimizers

TL;DR

This paper introduces a general recursive regularization method that transforms primal algorithms into dual solutions for convex-concave minimax problems, achieving improved runtimes and query complexities in various applications.

Contribution

The paper presents a novel recursive regularization technique that converts primal algorithms into dual solutions, enhancing efficiency in solving minimax and related optimization problems.

Findings

Achieved state-of-the-art runtimes for matrix games.

Improved query complexity for CVaR distributionally robust optimization.

Recovered optimal query complexity for stationary point finding.

Abstract

We provide a general method to convert a "primal" black-box algorithm for solving regularized convex-concave minimax optimization problems into an algorithm for solving the associated dual maximin optimization problem. Our method adds recursive regularization over a logarithmic number of rounds where each round consists of an approximate regularized primal optimization followed by the computation of a dual best response. We apply this result to obtain new state-of-the-art runtimes for solving matrix games in specific parameter regimes, obtain improved query complexity for solving the dual of the CVaR distributionally robust optimization (DRO) problem, and recover the optimal query complexity for finding a stationary point of a convex function.