Local adapt-then-combine algorithms for distributed nonsmooth optimization: Achieving provable communication acceleration

TL;DR

This paper introduces FlexATC, a communication-efficient distributed optimization framework that achieves provable acceleration and flexibility in communication, applicable to nonsmooth problems over networks.

Contribution

It proposes a unified ATC-based framework with provable communication acceleration and decoupled convergence rates, allowing communication skipping without performance loss.

Findings

FlexATC achieves sublinear and linear convergence in convex and strongly convex cases.

Communication skipping does not impair the linear convergence rate.

Numerical results confirm theoretical advantages of FlexATC.

Abstract

This paper is concerned with the distributed composite optimization problem over networks, where agents aim to minimize a sum of local smooth components and a common nonsmooth term. Leveraging the probabilistic local updates mechanism, we propose a communication-efficient Adapt-Then-Combine (ATC) framework, FlexATC, unifying numerous ATC-based distributed algorithms. Under stepsizes independent of the network topology and the number of local updates, we establish sublinear and linear convergence rates for FlexATC in convex and strongly convex settings, respectively. Remarkably, in the strong convex setting, the linear rate is decoupled from the objective functions and network topology, and FlexATC permits communication to be skipped in most iterations without any deterioration of the linear rate. In addition, the proposed unified theory demonstrates for the first time that local updates provably lead to communication acceleration for ATC-based distributed algorithms. Numerical experiments further validate the efficacy of the proposed framework and corroborate the theoretical results.