Two-timescale EXTRA for Distributed Smooth Non-convex Optimization

TL;DR

This paper introduces Two-timescale EXTRA, a novel distributed optimization algorithm for smooth non-convex problems, demonstrating sub-linear convergence and practical parameter selection methods.

Contribution

It proposes a new two-timescale variant of EXTRA with convergence guarantees and an off-line parameter selection strategy.

Findings

Proves sub-linear convergence to stationary points.

Develops an off-line method for parameter tuning.

Numerical results validate theoretical claims.

Abstract

In this paper, we study distributed optimization with smooth non-convex local objectives. We propose a novel variant of the well-known EXact firsT-ordeR Algorithm (EXTRA), called Two-timescale EXTRA, by introducing two distinct step-sizes. Leveraging the two-timescale strategy, we construct a Lyapunov function and establish the sub-linear convergence of Two-timescale EXTRA to a consensual first-order stationary point. Additionally, we introduce an off-line sequential method for algorithm parameter selection, and the numerical results support the theoretical guarantees.