BFGS-ADMM for Large-Scale Distributed Optimization

TL;DR

This paper introduces a novel distributed optimization algorithm combining quasi-Newton methods with ADMM, enabling efficient large-scale multi-agent cooperation with linear convergence guarantees and reduced communication overhead.

Contribution

It proposes a quasi-Newton ADMM approach with inexact primal updates and a block diagonal Hessian, improving convergence and communication efficiency in distributed settings.

Findings

Achieves global linear convergence without backtracking line search.

Reduces inner communication loops via an intermediate consensus variable.

Demonstrates superior performance over existing methods on real datasets.

Abstract

We consider a class of distributed optimization problem where the objective function consists of a sum of strongly convex and smooth functions and a (possibly nonsmooth) convex regularizer. A multi-agent network is assumed, where each agent holds a private cost function and cooperates with its neighbors to compute the optimum of the aggregate objective. We propose a quasi-Newton Alternating Direction Method of Multipliers (ADMM) where the primal update is solved inexactly with approximated curvature information. By introducing an intermediate consensus variable, we achieve a block diagonal Hessian which eliminates the need for inner communication loops within the network when computing the update direction. We establish global linear convergence to the optimal primal-dual solution without the need for backtracking line search, under the assumption that component cost functions are strongly convex with Lipschitz continuous gradients. Numerical simulations on real datasets demonstrate the advantages of the proposed method over state of the art.