Linear Convergence Rate Analysis of a Class of Exact First-Order Distributed Methods for Weight-Balanced Time-Varying Networks and Uncoordinated Step Sizes

TL;DR

This paper proves linear convergence of a broad class of exact first-order distributed algorithms over time-varying, weight-balanced directed networks with uncoordinated step sizes, extending existing methods' robustness and applicability.

Contribution

It introduces a unified analysis showing linear convergence of several existing algorithms under more general, realistic network and step-size conditions.

Findings

Establishes R-linear convergence under time-varying networks and uncoordinated step sizes.

Demonstrates robustness of unification and spectral step-size strategies to network variations.

Extends prior analyses to more general and practical distributed optimization settings.

Abstract

We analyze a class of exact distributed first order methods under a general setting on the underlying network and step-sizes. In more detail, we allow simultaneously for time-varying uncoordinated stepsizes and time-varying directed weight-balanced networks, jointly connected over bounded intervals. The analyzed class of methods subsumes several existing algorithms like the unified Extra and unified DIGing (Jakovetic, 2019), or the exact spectral gradient method (Jakovetic, Krejic, Krklec Jerinkic, 2019) that have been analyzed before under more restrictive assumptions. Under the assumed setting, we establish R-linear convergence of the methods and present several implications that our results have on the literature. Most notably, we show that the unification strategy in (Jakovetic, 2019) and the spectral step-size selection strategy in (Jakovetic, Krejic, Krklec Jerinkic, 2019) exhibit a high degree of robustness to uncoordinated time-varying step sizes and to time-varying networks.