Min-Max Optimization with Dual-Linear Coupling

TL;DR

This paper introduces a new algorithmic framework for convex-concave min-max problems with linear coupling, improving complexity bounds and combining randomization, extrapolation, and anchoring techniques in mirror descent methods.

Contribution

It develops a general algorithmic approach for coupled min-max problems with linear structure, offering improved complexity bounds and novel update strategies.

Findings

Potential complexity improvements over existing methods.

Effective combination of randomization, extrapolation, and anchoring.

Applicable to bilinear and nonbilinear coupled problems.

Abstract

We study a class of convex-concave min-max problems in which the coupled component of the objective is linear in at least one of the two decision vectors. We identify such problem structure as interpolating between the bilinearly and nonbilinearly coupled problems, motivated by key applications in areas such as distributionally robust optimization and convex optimization with functional constraints. Leveraging the considered nonlinear-linear coupling of the primal and the dual decision vectors, we develop a general algorithmic framework leading to fine-grained complexity bounds exploiting separability properties of the problem, whenever present. The obtained complexity bounds offer potential improvements over state-of-the-art scaling with $\sqrt{n}$ or $n$ in some of the considered problem settings, which even include bilinearly coupled problems, where $n$ is the dimension of the dual decision vector. On the algorithmic front, our work provides novel strategies for combining randomization with extrapolation and multi-point anchoring in the mirror descent-style updates in the primal and the dual, which we hope will find further applications in addressing related optimization problems. %