L-DQN: An Asynchronous Limited-Memory Distributed Quasi-Newton Method

TL;DR

L-DQN is a novel distributed asynchronous quasi-Newton algorithm for empirical risk minimization that offers efficient communication, low memory usage, and provable linear convergence guarantees in the presence of delays.

Contribution

This paper introduces L-DQN, the first distributed quasi-Newton method with proven global linear convergence under asynchronous delays.

Findings

Supports asynchronous computations among worker nodes.

Achieves efficient communication with vectors of size O(d).

Demonstrates practical performance through numerical experiments.

Abstract

This work proposes a distributed algorithm for solving empirical risk minimization problems, called L-DQN, under the master/worker communication model. L-DQN is a distributed limited-memory quasi-Newton method that supports asynchronous computations among the worker nodes. Our method is efficient both in terms of storage and communication costs, i.e., in every iteration the master node and workers communicate vectors of size $O(d)$, where $d$ is the dimension of the decision variable, and the amount of memory required on each node is $O(md)$, where $m$ is an adjustable parameter. To our knowledge, this is the first distributed quasi-Newton method with provable global linear convergence guarantees in the asynchronous setting where delays between nodes are present. Numerical experiments are provided to illustrate the theory and the practical performance of our method.