Asynchronous Distributed Optimization via Randomized Dual Proximal Gradient

TL;DR

This paper introduces asynchronous distributed optimization algorithms based on randomized dual proximal gradient methods, enabling efficient parallel updates without global synchronization for separable convex problems.

Contribution

It develops novel asynchronous algorithms for distributed optimization using dual proximal gradient methods with randomized block-coordinate updates, removing the need for global synchronization.

Findings

Algorithms are proven to be proper randomized block-coordinate proximal gradient updates.

Asynchronous updates are triggered by local timers, enhancing scalability.

The methods handle non-smooth, separable convex functions with shared variables.

Abstract

In this paper we consider distributed optimization problems in which the cost function is separable, i.e., a sum of possibly non-smooth functions all sharing a common variable, and can be split into a strongly convex term and a convex one. The second term is typically used to encode constraints or to regularize the solution. We propose a class of distributed optimization algorithms based on proximal gradient methods applied to the dual problem. We show that, by choosing suitable primal variable copies, the dual problem is itself separable when written in terms of conjugate functions, and the dual variables can be stacked into non-overlapping blocks associated to the computing nodes. We first show that a weighted proximal gradient on the dual function leads to a synchronous distributed algorithm with local dual proximal gradient updates at each node. Then, as main paper contribution, we develop asynchronous versions of the algorithm in which the node updates are triggered by local timers without any global iteration counter. The algorithms are shown to be proper randomized block-coordinate proximal gradient updates on the dual function.