A Universal Decomposition for Distributed Optimization Algorithms

TL;DR

This paper presents a universal decomposition framework that separates optimization and consensus tasks in distributed algorithms, enabling systematic design and analysis of multi-agent optimization methods.

Contribution

It introduces a universal decomposition of distributed optimization algorithms into centralized optimization and consensus components, facilitating new algorithm design approaches.

Findings

Any centralized optimization method can be combined with a consensus estimator for distributed implementation.

Many existing distributed algorithms can be expressed through this decomposition.

The framework suggests a systematic methodology for designing distributed optimization algorithms.

Abstract

In the distributed optimization problem for a multi-agent system, each agent knows a local function and must find a minimizer of the sum of all agents' local functions by performing a combination of local gradient evaluations and communicating information with neighboring agents. We prove that every distributed optimization algorithm can be factored into a centralized optimization method and a second-order consensus estimator, effectively separating the "optimization" and "consensus" tasks. We illustrate this fact by providing the decomposition for many recently proposed distributed optimization algorithms. Conversely, we prove that any optimization method that converges in the centralized setting can be combined with any second-order consensus estimator to form a distributed optimization algorithm that converges in the multi-agent setting. Finally, we describe how our decomposition may lead to a more systematic algorithm design methodology.