Decentralized optimization with affine constraints over time-varying networks

TL;DR

This paper introduces a novel decentralized optimization algorithm that handles affine constraints over dynamic networks, achieving optimal convergence rates and extending existing methods to more complex constraint scenarios.

Contribution

It presents the first linearly convergent decentralized algorithm for affine-constrained problems on time-varying networks, generalizing the ADOM algorithm.

Findings

Achieves linear convergence rate for decentralized affine-constrained optimization.

Proves the optimality of the convergence rate among first-order methods.

Provides lower bounds for communication and oracle calls in this setting.

Abstract

The decentralized optimization paradigm assumes that each term of a finite-sum objective is privately stored by the corresponding agent. Agents are only allowed to communicate with their neighbors in the communication graph. We consider the case when the agents additionally have local affine constraints and the communication graph can change over time. We provide the first linearly convergent decentralized algorithm for time-varying networks by generalizing the optimal decentralized algorithm ADOM to the case of affine constraints. We show that its rate of convergence is optimal for first-order methods by providing the lower bounds for the number of communications and oracle calls.