Dual Acceleration for Minimax Optimization: Linear Convergence Under Relaxed Assumptions

TL;DR

This paper introduces new algorithms for bilinear minimax problems that achieve linear convergence under weaker assumptions than existing methods, broadening applicability and improving theoretical guarantees.

Contribution

The paper proposes the PDPG and iDAPG algorithms, which attain linear convergence under relaxed conditions, advancing the state-of-the-art in minimax optimization.

Findings

PDPG converges linearly under weaker assumptions than prior algorithms.

iDAPG achieves linear convergence with even fewer restrictions.

iDAPG outperforms existing methods in certain scenarios.

Abstract

This paper addresses the bilinearly coupled minimax optimization problem: $\min_{x \in \mathbb{R}^{d_x}}\max_{y \in \mathbb{R}^{d_y}} \ f_1(x) + f_2(x) + y^{\top} Bx - g_1(y) - g_2(y)$, where $f_1$ and $g_1$ are smooth convex functions, $f_2$ and $g_2$ are potentially nonsmooth convex functions, and $B$ is a coupling matrix. Existing algorithms for solving this problem achieve linear convergence only under stronger conditions, which may not be met in many scenarios. We first introduce the Primal-Dual Proximal Gradient (PDPG) method and demonstrate that it converges linearly under an assumption where existing algorithms fail to achieve linear convergence. Building on insights gained from analyzing the convergence conditions of existing algorithms and PDPG, we further propose the inexact Dual Accelerated Proximal Gradient (iDAPG) method. This method achieves linear convergence under weaker conditions than those required by existing approaches. Moreover, even in cases where existing methods guarantee linear convergence, iDAPG can still provide superior theoretical performance in certain scenarios.