A Primal-Dual Quasi-Newton Method for Exact Consensus Optimization

TL;DR

This paper presents a primal-dual quasi-Newton method for decentralized consensus optimization that achieves exact solutions with linear convergence, combining second-order approximation with decentralized updates for improved robustness and efficiency.

Contribution

It introduces a novel primal-dual quasi-Newton algorithm that approximates second-order information in a fully decentralized manner for consensus optimization.

Findings

Achieves linear convergence rate.

Reduces computational burden compared to dual methods.

Demonstrates strong performance advantages numerically.

Abstract

We introduce the primal-dual quasi-Newton (PD-QN) method as an approximated second order method for solving decentralized optimization problems. The PD-QN method performs quasi-Newton updates on both the primal and dual variables of the consensus optimization problem to find the optimal point of the augmented Lagrangian. By optimizing the augmented Lagrangian, the PD-QN method is able to find the exact solution to the consensus problem with a linear rate of convergence. We derive fully decentralized quasi-Newton updates that approximate second order information to reduce the computational burden relative to dual methods and to make the method more robust in ill-conditioned problems relative to first order methods. The linear convergence rate of PD-QN is established formally and strong performance advantages relative to existing dual and primal-dual methods are shown numerically.