A Dual Approach for Optimal Algorithms in Distributed Optimization over Networks

TL;DR

This paper develops dual-based distributed algorithms for convex optimization over networks, achieving near-centralized optimal rates with complexity bounds tailored to different function properties and network structures.

Contribution

It introduces a unified dual approach for distributed convex optimization, providing complexity bounds and algorithms that match centralized rates under various function assumptions.

Findings

Algorithms achieve optimal convergence rates similar to centralized methods.

Complexity bounds depend on the spectral properties of the network.

Numerical experiments validate the effectiveness of the proposed methods.

Abstract

We study dual-based algorithms for distributed convex optimization problems over networks, where the objective is to minimize a sum $\sum_{i=1}^{m}f_i(z)$ of functions over in a network. We provide complexity bounds for four different cases, namely: each function $f_i$ is strongly convex and smooth, each function is either strongly convex or smooth, and when it is convex but neither strongly convex nor smooth. Our approach is based on the dual of an appropriately formulated primal problem, which includes a graph that models the communication restrictions. We propose distributed algorithms that achieve the same optimal rates as their centralized counterparts (up to constant and logarithmic factors), with an additional optimal cost related to the spectral properties of the network. Initially, we focus on functions for which we can explicitly minimize its Legendre-Fenchel conjugate, i.e., admissible or dual friendly functions. Then, we study distributed optimization algorithms for non-dual friendly functions, as well as a method to improve the dependency on the parameters of the functions involved. Numerical analysis of the proposed algorithms is also provided.