A Primal-Dual Parallel Method with $O(1/\epsilon)$ Convergence for Constrained Composite Convex Programs

TL;DR

This paper introduces a new primal-dual parallel algorithm that achieves an $O(1/ ext{epsilon})$ convergence rate for large-scale constrained convex programs, even when the problem is composite and non-separable, with low per-iteration complexity.

Contribution

It proposes a novel primal-dual method that maintains fast convergence and parallelizability for complex constrained convex programs, overcoming limitations of previous methods.

Findings

Achieves $O(1/ ext{epsilon})$ convergence rate.

Supports parallel implementation with low complexity per iteration.

Handles non-separable, composite convex programs effectively.

Abstract

This paper considers large scale constrained convex (possibly composite and non-separable) programs, which are usually difficult to solve by interior point methods or other Newton-type methods due to the non-smoothness or the prohibitive computation and storage complexity for Hessians and matrix inversions. Instead, they are often solved by first order gradient based methods or decomposition based methods. The conventional primal-dual subgradient method, also known as the Arrow-Hurwicz-Uzawa subgradient method, is a low complexity algorithm with an $O(1/\epsilon^2)$ convergence time. Recently, a new Lagrangian dual type algorithm with a faster $O(1/\epsilon)$ convergence time is proposed in Yu and Neely (2017). However, if the objective or constraint functions are not separable, each iteration of the Lagrangian dual type method in Yu and Neely (2017) requires to solve a unconstrained convex program, which can have huge complexity. This paper proposes a new primal-dual type algorithm with $O(1/\epsilon)$ convergence for general constrained convex programs. Each iteration of the new algorithm can be implemented in parallel with low complexity even when the original problem is composite and non-separable.