A General Framework for Distributed Partitioned Optimization

TL;DR

This paper introduces a versatile framework for decentralized partitioned optimization that reduces communication load and achieves accelerated convergence by leveraging a generalized Laplacian matrix, applicable to large-scale and privacy-sensitive systems.

Contribution

It presents a novel, algorithm-independent framework for decentralized optimization with variable-dependent local functions, enabling efficient algorithms with explicit convergence rates.

Findings

Framework reduces communication in decentralized systems.

Achieves non-asymptotic convergence with explicit network dependence.

Demonstrated effectiveness on synthetic examples.

Abstract

Decentralized optimization is widely used in large scale and privacy preserving machine learning and various distributed control and sensing systems. It is assumed that every agent in the network possesses a local objective function, and the nodes interact via a communication network. In the standard scenario, which is mostly studied in the literature, the local functions are dependent on a common set of variables, and, therefore, have to send the whole variable set at each communication round. In this work, we study a different problem statement, where each of the local functions held by the nodes depends only on some subset of the variables. Given a network, we build a general algorithm-independent framework for decentralized partitioned optimization that allows to construct algorithms with reduced communication load using a generalization of Laplacian matrix. Moreover, our framework allows to obtain algorithms with non-asymptotic convergence rates with explicit dependence on the parameters of the network, including accelerated and optimal first-order methods. We illustrate the efficacy of our approach on a synthetic example.