Linear Convergence Rate of a Class of Distributed Augmented Lagrangian Algorithms

TL;DR

This paper proves that a class of distributed augmented Lagrangian algorithms converge at a linear rate under certain smoothness conditions, with explicit dependence on network parameters, supported by simulations.

Contribution

It establishes the first explicit linear convergence rates for distributed AL methods with detailed network dependence, enhancing theoretical understanding.

Findings

Distributed AL methods achieve linear convergence under smoothness assumptions.

Explicit convergence rate formulas depend on network parameters.

Simulations confirm theoretical predictions.

Abstract

We study distributed optimization where nodes cooperatively minimize the sum of their individual, locally known, convex costs $f_i(x)$'s, $x \in {\mathbb R}^d$ is global. Distributed augmented Lagrangian (AL) methods have good empirical performance on several signal processing and learning applications, but there is limited understanding of their convergence rates and how it depends on the underlying network. This paper establishes globally linear (geometric) convergence rates of a class of deterministic and randomized distributed AL methods, when the $f_i$'s are twice continuously differentiable and have a bounded Hessian. We give explicit dependence of the convergence rates on the underlying network parameters. Simulations illustrate our analytical findings.