Contractivity and linear convergence in bilinear saddle-point problems: An operator-theoretic approach

TL;DR

This paper uses operator theory to analyze the convergence of primal-dual algorithms for bilinear saddle-point problems, providing new guarantees and tighter bounds especially when functions are not strongly convex.

Contribution

It introduces an operator-theoretic framework to establish contractivity and linear convergence of primal-dual algorithms under various convexity and rank conditions.

Findings

Proves contractivity of primal-dual algorithms using monotone operator theory.

Provides new convergence guarantees and tighter bounds than existing results.

Applies to problems with varying convexity conditions on functions f and g.

Abstract

We study the convex-concave bilinear saddle-point problem $\min_x \max_y f(x) + y^\top Ax - g(y)$, where both, only one, or none of the functions $f$ and $g$ are strongly convex, and suitable rank conditions on the matrix $A$ hold. The solution of this problem is at the core of many machine learning tasks. By employing tools from monotone operator theory, we systematically prove the contractivity (in turn, the linear convergence) of several first-order primal-dual algorithms, including the Chambolle-Pock method. Our approach results in concise proofs, and it yields new convergence guarantees and tighter bounds compared to known results.