On the Linear Convergence of Distributed Optimization over Directed Graphs

TL;DR

This paper introduces DEXTRA, a distributed algorithm that achieves linear convergence over directed graphs for strongly-convex optimization problems, outperforming existing methods with sublinear rates.

Contribution

The paper presents DEXTRA, a novel distributed algorithm that guarantees linear convergence over directed graphs for strongly-convex functions, improving upon previous sublinear convergence rates.

Findings

DEXTRA converges linearly at rate O(τ^k) for strongly-convex functions.

Existing algorithms have sublinear convergence rates of O(ln k/√k) or O(ln k/k).

Simulation results confirm the theoretical convergence rate.

Abstract

This paper develops a fast distributed algorithm, termed \emph{DEXTRA}, to solve the optimization problem when~$n$ agents reach agreement and collaboratively minimize the sum of their local objective functions over the network, where the communication between the agents is described by a~\emph{directed} graph. Existing algorithms solve the problem restricted to directed graphs with convergence rates of $O(\ln k/\sqrt{k})$ for general convex objective functions and $O(\ln k/k)$ when the objective functions are strongly-convex, where~$k$ is the number of iterations. We show that, with the appropriate step-size, DEXTRA converges at a linear rate $O(\tau^{k})$ for $0<\tau<1$, given that the objective functions are restricted strongly-convex. The implementation of DEXTRA requires each agent to know its local out-degree. Simulation examples further illustrate our findings.