Decentralized Approximate Newton Methods for Convex Optimization on Networked Systems

TL;DR

This paper introduces Decentralized Approximate Newton (DEAN) methods enabling networked nodes to collaboratively solve convex optimization problems efficiently using local computations, with proven convergence properties and competitive simulation results.

Contribution

The paper develops a novel DEAN algorithm that combines local approximate Newton updates with global consensus tracking, improving convergence under weaker assumptions.

Findings

Nodes can reach near-optimal consensus with less restrictive assumptions.

The DEAN algorithm achieves linear convergence to a suboptimal solution.

Simulations show DEAN's superior speed and accuracy compared to existing methods.

Abstract

In this paper, a class of Decentralized Approximate Newton (DEAN) methods for addressing convex optimization on a networked system are developed, where nodes in the networked system seek for a consensus that minimizes the sum of their individual objective functions through local interactions only. The proposed DEAN algorithms allow each node to repeatedly perform a local approximate Newton update, which leverages tracking the global Newton direction and dissipating the discrepancies among the nodes. Under less restrictive problem assumptions in comparison with most existing second-order methods, the DEAN algorithms enable the nodes to reach a consensus that can be arbitrarily close to the optimum. Moreover, for a particular DEAN algorithm, the nodes linearly converge to a common suboptimal solution with an explicit error bound. Finally, simulations demonstrate the competitive performance of DEAN in convergence speed, accuracy, and efficiency.