An Asynchronous Distributed Proximal Gradient Method for Composite Convex Optimization

TL;DR

This paper introduces an asynchronous distributed proximal gradient method for convex optimization that efficiently solves large-scale problems with communication constraints, demonstrating theoretical convergence and practical effectiveness.

Contribution

It develops a novel asynchronous distributed proximal gradient algorithm with convergence guarantees for composite convex functions, extending prior synchronous methods.

Findings

Converges to an optimal solution with $ ilde{O}(rac{ ext{eigmax}^{1.5}}{d_{min}}rac{1}{ ext{epsilon}})$ complexity.

Achieves $ ext{epsilon}$-optimal and feasible solutions within $ ilde{O}( ext{log}(rac{1}{ ext{epsilon}}))$ iterations.

Demonstrates efficiency on large-scale sparse-group LASSO problems.

Abstract

We propose a distributed first-order augmented Lagrangian (DFAL) algorithm to minimize the sum of composite convex functions, where each term in the sum is a private cost function belonging to a node, and only nodes connected by an edge can directly communicate with each other. This optimization model abstracts a number of applications in distributed sensing and machine learning. We show that any limit point of DFAL iterates is optimal; and for any $\epsilon>0$, an $\epsilon$-optimal and $\epsilon$-feasible solution can be computed within $\mathcal{O}(\log(\epsilon^{-1}))$ DFAL iterations, which require $\mathcal{O}(\frac{\psi_{\max}^{1.5}}{d_{\min}} \epsilon^{-1})$ proximal gradient computations and communications per node in total, where $\psi_{\max}$ denotes the largest eigenvalue of the graph Laplacian, and $d_{\min}$ is the minimum degree of the graph. We also propose an asynchronous version of DFAL by incorporating randomized block coordinate descent methods; and demonstrate the efficiency of DFAL on large scale sparse-group LASSO problems.