ADD-OPT: Accelerated Distributed Directed Optimization

TL;DR

This paper introduces ADD-OPT, a new algorithm for distributed optimization over directed networks that achieves the fastest convergence rates and supports flexible step-size choices, outperforming existing methods.

Contribution

We propose ADD-OPT, the first algorithm with optimal convergence rate for directed networks and broad step-size support in distributed strongly-convex optimization.

Findings

Achieves the best known convergence rate of O(μ^k) for directed networks.

Supports a wider range of step-sizes, including arbitrarily small positive values.

Demonstrates superior performance through simulations.

Abstract

In this paper, we consider distributed optimization problems where the goal is to minimize a sum of objective functions over a multi-agent network. We focus on the case when the inter-agent communication is described by a strongly-connected, \emph{directed} graph. The proposed algorithm, ADD-OPT (Accelerated Distributed Directed Optimization), achieves the best known convergence rate for this class of problems,~$O(\mu^{k}),0<\mu<1$, given strongly-convex, objective functions with globally Lipschitz-continuous gradients, where~$k$ is the number of iterations. Moreover, ADD-OPT supports a wider and more realistic range of step-sizes in contrast to existing work. In particular, we show that ADD-OPT converges for arbitrarily small (positive) step-sizes. Simulations further illustrate our results.