Convergence Analysis via ODE Approach for Convex Optimization with Linear Equality Constraints

TL;DR

This paper analyzes the convergence of primal-dual algorithms for linearly constrained convex optimization using an ODE approach, extending classical methods to nonsmooth, large-scale problems with explicit convergence rates.

Contribution

It extends the ODE-based convergence analysis to nonsmooth, nonstrongly convex problems with linear constraints, incorporating geometric conditions for explicit convergence rates.

Findings

Established an $ ext{O}(1/t^2)$ local convergence rate.

Derived a numerical scheme by discretizing the ODE.

Provided insights into optimal parameter selection.

Abstract

This paper studies the continuous-time dynamics of primal-dual algorithms for linearly constrained convex optimization problems and provides a quantitative convergence analysis using the Lyapunov functions. With the growing prevalence of sparse and low-rank models, large-scale problems involving nonsmooth objective functions have become increasingly important. Our approach addresses nonsmooth and nonstrong convex objective functions, which is particularly effective in extending classical accelerated methods to broader large-scale optimization problems. Building upon the ordinary differential equation (ODE) approach inspired by the recent work on Nesterov's acceleration methods, we extend the analysis to an ODE associated with an optimization problem with linear equality constraints. Moreover, by imposing a geometric condition analogous to the \textit{Kurdyka--\text{\L}ojasiewicz (K\text{\L}) property} on the objective function, we derive convergence rates that depend explicitly on the local geometry and establish the $\mathcal{O}(1/t^2)$ local convergence rate. For the algorithmic construction, a numerical scheme is derived by discretizing the proposed ODE. Furthermore, we investigate the influence of algorithm parameters and provide insights into their optimal selection. Finally, preliminary numerical experiments are provided to validate the consistency with the theoretical results.