Decentralized Optimization with Mixed Affine Constraints

TL;DR

This paper develops optimal algorithms for decentralized convex optimization with mixed affine constraints, addressing both smooth and non-smooth cases, and matching theoretical lower bounds in convergence rates.

Contribution

It introduces an optimal algorithm for smooth, strongly convex problems with mixed constraints and provides near-optimal methods for other cases, advancing decentralized optimization techniques.

Findings

Optimal convergence rate for smooth, strongly convex case.

Near-optimal methods for non-smooth and general convex cases.

Applicability to federated learning and resource allocation.

Abstract

This paper considers decentralized optimization of convex functions with mixed affine equality constraints involving both local and global variables. Constraints on global variables may vary across different nodes in the network, while local variables are subject to coupled and node-specific constraints. Such problem formulations arise in machine learning applications, including federated learning and multi-task learning, as well as in resource allocation and distributed control. We analyze this problem under smooth and non-smooth assumptions, considering both strongly convex and general convex objective functions. Our main contribution is an optimal algorithm for the smooth, strongly convex regime, whose convergence rate matches established lower complexity bounds. We further provide near-optimal methods for the remaining cases.