Totally Asynchronous Primal-Dual Convex Optimization in Blocks

TL;DR

This paper introduces a robust parallel primal-dual convex optimization algorithm that handles various asynchronous updates and communications without delay bounds, providing convergence guarantees and practical insights.

Contribution

It develops an asynchronous primal-dual algorithm for convex optimization, analyzing communication delays, and establishing convergence rates without delay bounds.

Findings

Algorithm is robust to multiple forms of asynchrony.

Convergence rates include an asynchrony penalty.

Numerical results validate theoretical findings.

Abstract

We present a parallelized primal-dual algorithm for solving constrained convex optimization problems. The algorithm is "block-based," in that vectors of primal and dual variables are partitioned into blocks, each of which is updated only by a single processor. We consider four possible forms of asynchrony: in updates to primal variables, updates to dual variables, communications of primal variables, and communications of dual variables. We construct a family of explicit counterexamples to show the need to eliminate asynchronous communication of dual variables, though the other forms of asynchrony are permitted, all without requiring bounds on delays. A first-order primal-dual update law is developed and shown to be robust to asynchrony. We then derive convergence rates to a Lagrangian saddle point in terms of the operations agents execute, without specifying any timing or pattern with which they must be executed. These convergence rates include an "asynchrony penalty" that we quantify and present ways to mitigate. Numerical results illustrate these developments.