DIPPA: An improved Method for Bilinear Saddle Point Problems

TL;DR

This paper introduces a new algorithm for bilinear saddle point problems that only requires gradient access, achieving optimal complexity bounds and improving applicability in scenarios where proximal operators are hard to compute.

Contribution

The work proposes a gradient-only algorithm for bilinear saddle point problems with optimal complexity bounds, avoiding the need for proximal operators.

Findings

Achieves complexity upper bound with optimal dependency on the coupling condition number.

Applicable to constraint zero-sum matrix games where proximal operators are difficult to compute.

Improves efficiency over existing methods that require proximal oracle access.

Abstract

This paper studies bilinear saddle point problems $\min_{\bf{x}} \max_{\bf{y}} g(\bf{x}) + \bf{x}^{\top} \bf{A} \bf{y} - h(\bf{y})$, where the functions $g, h$ are smooth and strongly-convex. When the gradient and proximal oracle related to $g$ and $h$ are accessible, optimal algorithms have already been developed in the literature \cite{chambolle2011first, palaniappan2016stochastic}. However, the proximal operator is not always easy to compute, especially in constraint zero-sum matrix games \cite{zhang2020sparsified}. This work proposes a new algorithm which only requires the access to the gradients of $g, h$. Our algorithm achieves a complexity upper bound $\tilde{\mathcal{O}}\left( \frac{\|\bf{A}\|_2}{\sqrt{\mu_x \mu_y}} + \sqrt[4]{\kappa_x \kappa_y (\kappa_x + \kappa_y)} \right)$ which has optimal dependency on the coupling condition number $\frac{\|\bf{A}\|_2}{\sqrt{\mu_x \mu_y}}$ up to logarithmic factors.