A Line-search-free Method for Adaptive Decentralized Optimization

TL;DR

This paper introduces a decentralized optimization method that adapts stepsizes without line-searches or global parameters, ensuring efficient convergence for convex problems.

Contribution

It presents a novel line-search-free, fully decentralized algorithm with adaptive stepsizes based on local curvature estimates, eliminating the need for global tuning.

Findings

Achieves sublinear convergence for convex objectives.

Attains linear convergence under strong convexity.

Demonstrates improved performance over existing methods in experiments.

Abstract

We study decentralized optimization over networks where agents cooperatively minimize a smooth (strongly) convex sum of local losses while communicating only with immediate neighbors. Prevailing decentralized methods require either centralized knowledge of global problem and network parameters for stepsize tuning--hence impractical, or costly per-iteration line-searches that demand access to local function values. We propose line-search-free, fully decentralized algorithms in which each agent adapts its stepsize using only past local iterates and gradients--with no extra function evaluations and no global tuning. The key technical ingredient is a new Lyapunov function, from which a natural adaptive stepsize rule emerges: at each iteration, each agent selects the largest stepsize that guarantees descent, based solely on a local curvature estimate built from successive gradients. The proposed algorithms enjoy strong theoretical guarantees: sublinear convergence rates for merely convex objectives and linear rates under strong convexity. Numerical experiments on standard benchmarks show consistent improvements over the state of the art, both adaptive and non-adaptive.