Randomized dual proximal gradient for large-scale distributed optimization

TL;DR

This paper introduces a randomized dual proximal gradient method for large-scale distributed optimization, enabling asynchronous updates over networks with separable, possibly non-smooth, convex functions.

Contribution

It proposes a novel asynchronous distributed algorithm based on randomized block-coordinate proximal gradient on the dual problem, handling non-smooth separable functions.

Findings

Algorithm converges under specified conditions.

Enables efficient distributed optimization over undirected networks.

Handles non-smooth regularization and constraints.

Abstract

In this paper we consider distributed optimization problems in which the cost function is separable (i.e., a sum of possibly non-smooth functions all sharing a common variable) and can be split into a strongly convex term and a convex one. The second term is typically used to encode constraints or to regularize the solution. We propose an asynchronous, distributed optimization algorithm over an undirected topology, based on a proximal gradient update on the dual problem. We show that by means of a proper choice of primal variables, the dual problem is separable and the dual variables can be stacked into separate blocks. This allows us to show that a distributed gossip update can be obtained by means of a randomized block-coordinate proximal gradient on the dual function.