Communication Efficient Curvature Aided Primal-dual Algorithms for Decentralized Optimization

TL;DR

This paper introduces a family of communication-efficient decentralized algorithms that leverage curvature information for convex optimization, achieving faster convergence and asynchronous implementation.

Contribution

It proposes a unified framework for curvature-aided primal-dual algorithms with convergence guarantees and an asynchronous variant for decentralized convex optimization.

Findings

Achieves sublinear and linear convergence rates depending on convexity assumptions.

Explicitly characterizes acceleration due to curvature information.

Demonstrates effectiveness through numerical experiments on benchmark datasets.

Abstract

This paper presents a family of algorithms for decentralized convex composite problems. We consider the setting of a network of agents that cooperatively minimize a global objective function composed of a sum of local functions plus a regularizer. Through the use of intermediate consensus variables, we remove the need for inner communication loops between agents when computing curvature-guided updates. A general scheme is presented which unifies the analysis for a plethora of computing choices, including gradient descent, Newton updates, and BFGS updates. Our analysis establishes sublinear convergence rates under convex objective functions with Lipschitz continuous gradients, as well as linear convergence rates when the local functions are further assumed to be strongly convex. Moreover, we explicitly characterize the acceleration due to curvature information. Last but not the least, we present an asynchronous implementation for the proposed algorithms, which removes the need for a central clock, with linear convergence rates established in expectation under strongly convex objectives. We ascertain the effectiveness of the proposed methods with numerical experiments on benchmark datasets.