Geometrically Convergent Distributed Optimization with Uncoordinated Step-Sizes

TL;DR

This paper proves that the ATC variation of the DIGing distributed optimization algorithm converges geometrically with uncoordinated step-sizes and can outperform distributed gradient descent in convergence speed.

Contribution

It demonstrates that the ATC DIGing algorithm converges geometrically with different step-sizes among agents, extending prior work requiring uniform step-sizes.

Findings

Convergence is geometric even with uncoordinated step-sizes.

ATC structure accelerates convergence compared to DGD.

Theoretical analysis supports faster convergence rates.

Abstract

A recent algorithmic family for distributed optimization, DIGing's, have been shown to have geometric convergence over time-varying undirected/directed graphs. Nevertheless, an identical step-size for all agents is needed. In this paper, we study the convergence rates of the Adapt-Then-Combine (ATC) variation of the DIGing algorithm under uncoordinated step-sizes. We show that the ATC variation of DIGing algorithm converges geometrically fast even if the step-sizes are different among the agents. In addition, our analysis implies that the ATC structure can accelerate convergence compared to the distributed gradient descent (DGD) structure which has been used in the original DIGing algorithm.