A Unified Control-Theoretic Framework for Saddle-Point Dynamics in Constrained Optimization

TL;DR

This paper presents a control-theoretic framework for saddle-point dynamics in constrained optimization, unifying classical methods and analyzing convergence properties using feedback control principles.

Contribution

It introduces a PID-based saddle-point flow framework that generalizes existing primal-dual methods and provides global exponential convergence guarantees for convex problems.

Findings

Equilibria of the flow match stationary points of the original problem.

Integral action enforces constraint satisfaction effectively.

Explicit bounds on convergence rates are derived for convex affine problems.

Abstract

This paper studies equality-constrained minimization problems through the lens of feedback control. We introduce a unified control-theoretic framework by showing that a PID feedback law acting on the dual variable induces the PID saddle-point flow (PID-SPF), a broad class of saddle-point dynamics associated with the augmented Lagrangian. This framework recovers several classical primal-dual flows as special cases. We prove that the equilibria of the proposed flow coincide with the stationary points of the original problem. Our analysis reveals how the feedback gains affect the optimization: integral action enforces constraint satisfaction, proportional action introduces the augmented Lagrangian structure, and derivative action modifies the geometry of the primal dynamics by inducing a state-dependent Riemannian metric. Moreover, for convex problems with affine constraints, we establish global exponential convergence by leveraging contraction theory for all admissible PID gains, providing in the process explicit bounds on the convergence rate. Finally, we validate our theoretical results on numerical examples including an application to bilevel optimization.