Primal-dual dynamical systems with closed-loop control for convex optimization in continuous and discrete time

TL;DR

This paper introduces a novel primal-dual dynamical system with closed-loop control for convex optimization, achieving fast convergence and adaptive step sizes, with demonstrated practical effectiveness.

Contribution

It develops a new continuous-time primal-dual system with feedback control and derives an accelerated discrete algorithm with adaptive step size for convex optimization.

Findings

Achieves fast convergence rates for primal-dual gap, feasibility, and objective residual.

Provides an accelerated primal-dual algorithm with adaptive gradient-based step size.

Numerical results show superior performance and practical efficacy.

Abstract

This paper develops a primal-dual dynamical system where the coefficients are designed in closed-loop way for solving a convex optimization problem with linear equality constraints. We first introduce a ``second-order primal" + ``first-order dual'' continuous-time dynamical system, in which both the time scaling and Hessian-driven damping are governed by a feedback control of the gradient for the Lagrangian function. This system achieves the fast convergence rates for the primal-dual gap, the feasibility violation, and the objective residual along its trajectory. Subsequently, by time discretization of this system, we develop an accelerated primal-dual algorithm with a gradient-defined adaptive step size. We also obtain convergence rates for the primal-dual gap, the feasibility violation, and the objective residual. Furthermore, we provide numerical results to demonstrate the practical efficacy and superior performance of the proposed algorithm.