A Fully Parallel Primal-Dual Algorithm for Centralized and Distributed Optimization

TL;DR

This paper introduces a fully parallel primal-dual algorithm for centralized and distributed optimization that updates all variables simultaneously, ensuring convergence even in multi-block scenarios, unlike some existing methods.

Contribution

The paper presents a novel parallel primal-dual algorithm that extends to multi-block optimization with guaranteed convergence, unlike existing sequential update methods.

Findings

The proposed algorithm converges for multi-block optimization.

It outperforms existing algorithms like ADMM in parallel settings.

Numerical example demonstrates convergence in a three-block problem.

Abstract

In this paper, a centralized two-block separable optimization is considered for which a fully parallel primal-dual discrete-time algorithm with fixed step size is derived based on monotone operator splitting method. In this algorithm, the primal variables are updated in an alternating fashion like Alternating Direction Method of Multipliers (ADMM). However, unlike existing discrete-time algorithms such as Method of Multipliers (MM), ADMM, Bi-Alternating Direction Method of Multipliers (BiADMM), and Primal-Dual Fixed Point (PDFP) algorithms, that all suffer from sequential updates, all primal and dual variables are updated in parallel in the sense that to update a variable at each time, updated version of other variable(s) is not required. One of advantages of the proposed algorithm is that its direct extension to multi-block optimization is still convergent. Then the method is applied to distributed optimization for which a fully parallel primal-dual distributed algorithm is obtained. Finally, since direct extension of ADMM may diverge for multi-block optimization, a numerical example of a three-block optimization is given for which the direct extension of the proposed algorithm is shown to converge to a solution.