Direct Spectral Acceleration of First-Order Methods for Saddle Point Problems with Bilinear Coupling

TL;DR

This paper introduces a spectral acceleration technique for first-order methods solving saddle point problems with bilinear coupling, achieving optimal linear convergence in various settings.

Contribution

It proposes a novel spectral acceleration method for primal-dual algorithms that attains optimal linear rates without complex inner loops or double loops.

Findings

Achieves optimal linear convergence rates in bilinear saddle point problems.

Develops stochastic block-coordinate extensions with matching lower bounds.

Provides explicit hard instances demonstrating worst-case optimality.

Abstract

We study convex-concave saddle point problems with bilinear coupling, covering linearly constrained convex optimization and more general nonsmooth or constrained models via a proximable term in the dual objective. In linearly convergent regimes, we characterize how spectral properties of the coupling matrix and objective conditioning jointly determine the attainable linear rates. We propose direct spectral acceleration for first-order primal--dual methods for a class of bilinear-coupled saddle point problems, including affinely constrained smooth strongly convex optimization and extensions with proximable dual terms. The resulting algorithms distinguish objective-dominated and coupling matrix-dominated regimes and attain optimal linear convergence without Chebyshev inner loops or double-loop designs. We further develop stochastic block-coordinate extensions in the affinely constrained case with separable objectives; we also establish optimal linear rates matching the block-coordinate lower bound. For both deterministic and stochastic methods, we provide matching worst-case lower bounds via explicit finite-dimensional hard instances.