Stability of Primal-Dual Gradient Flow Dynamics for Multi-Block Convex Optimization Problems

TL;DR

This paper analyzes the stability of primal-dual gradient flow dynamics for multi-block convex optimization, proposing a systematic approach with weaker assumptions than existing methods, and demonstrating exponential convergence and practical benefits.

Contribution

It introduces a systematic primal-dual gradient flow method with weaker assumptions, ensuring global exponential stability for multi-block convex problems, outperforming traditional ADMM and related algorithms.

Findings

Establishes global exponential convergence guarantees.

Provides weaker structural assumptions than existing methods.

Demonstrates effectiveness in parallel and distributed computing.

Abstract

We examine stability properties of primal-dual gradient flow dynamics for composite convex optimization problems with multiple, possibly nonsmooth, terms in the objective function under the generalized consensus constraint. The proposed dynamics are based on the proximal augmented Lagrangian and they provide a viable alternative to ADMM which faces significant challenges from both analysis and implementation viewpoints in large-scale multi-block scenarios. In contrast to customized algorithms with individualized convergence guarantees, we develop a systematic approach for solving a broad class of challenging composite optimization problems. We leverage various structural properties to establish global (exponential) convergence guarantees for the proposed dynamics. Our assumptions are much weaker than those required to prove (exponential) stability of primal-dual dynamics as well as (linear) convergence of discrete-time methods such as standard two-block and multi-block ADMM and EXTRA algorithms. Finally, we show necessity of some of our structural assumptions for exponential stability and provide computational experiments to demonstrate the convenience of the proposed approach for parallel and distributed computing applications.