Communication efficient quasi-Newton distributed optimization based on the Douglas-Rachford envelope

TL;DR

This paper introduces a communication-efficient distributed optimization method using Douglas-Rachford splitting and BFGS, achieving superlinear convergence with minimal communication, suitable for client-server settings.

Contribution

It presents a novel BFGS-based distributed optimization algorithm that reduces communication to one vector per round and does not require line search, with proven superlinear convergence.

Findings

Reduces communication cost significantly compared to existing methods.

Achieves superlinear convergence without line search.

Demonstrates effectiveness through experiments in decreasing communication and computation costs.

Abstract

We consider distributed optimization in the client-server setting. By use of Douglas-Rachford splitting to the dual of the sum problem, we design a BFGS method that requires minimal communication (sending/receiving one vector per round for each client). Our method is line search free and achieves superlinear convergence. Experiments are also used to demonstrate the merits in decreasing communication and computation costs.