Accelerated Primal-Dual Methods for Convex-Strongly-Concave Saddle Point Problems

TL;DR

This paper introduces an accelerated primal-dual method for convex-strongly-concave saddle point problems that achieves optimal gradient complexity, improving efficiency over existing methods especially for problems with nonlinear coupling.

Contribution

It proposes the ALPD method combining acceleration with linearized primal-dual techniques, achieving optimal complexity for semi-linear coupling and improved evaluations for nonlinear cases.

Findings

ALPD attains optimal gradient complexity for semi-linear coupling.

Inexact ALPD reduces gradient evaluations for nonlinear coupling.

Numerical experiments confirm theoretical improvements.

Abstract

We investigate a primal-dual (PD) method for the saddle point problem (SPP) that uses a linear approximation of the primal function instead of the standard proximal step, resulting in a linearized PD (LPD) method. For convex-strongly concave SPP, we observe that the LPD method has a suboptimal dependence on the Lipschitz constant of the primal function. To fix this issue, we combine features of Accelerated Gradient Descent with the LPD method resulting in a single-loop Accelerated Linearized Primal-Dual (ALPD) method. ALPD method achieves the optimal gradient complexity when the SPP has a semi-linear coupling function. We also present an inexact ALPD method for SPPs with a general nonlinear coupling function that maintains the optimal gradient evaluations of the primal parts and significantly improves the gradient evaluations of the coupling term compared to the ALPD method. We verify our findings with numerical experiments.