A Unified Analysis of Saddle Flow Dynamics: Stability and Algorithm Design

TL;DR

This paper provides a comprehensive analysis of saddle flow dynamics, establishing new convergence conditions, stability guarantees, and algorithmic insights for convex-concave optimization problems, with applications demonstrated on network flow and Lasso regression.

Contribution

It introduces a novel observability-based convergence certificate, a state-augmentation method, and generalizes stability results to broader classes of saddle flows and algorithms.

Findings

Proposes a certificate for asymptotic convergence based on observability.

Shows exponential stability under strong convexity-concavity.

Validates methods on network flow and Lasso regression problems.

Abstract

This work examines the conditions for asymptotic and exponential convergence of saddle flow dynamics of convex-concave functions. First, we propose an observability-based certificate for asymptotic convergence, directly bridging the gap between the invariant set in a LaSalle argument and the equilibrium set of saddle flows. This certificate generalizes conventional conditions for convergence, e.g., strict convexity-concavity, and leads to a novel state-augmentation method that requires minimal assumptions for asymptotic convergence. We also show that global exponential stability follows from strong convexity-strong concavity, providing a lower-bound estimate of the convergence rate. This insight also explains the convergence of proximal saddle flows for strongly convex-concave objective functions. Our results generalize to dynamics with projections on the vector field and have applications in solving constrained convex optimization via primal-dual methods. Based on these insights, we study four algorithms built upon different Lagrangian function transformations. We validate our work by applying these methods to solve a network flow optimization and a Lasso regression problem.