Distributed Approximate Newton Algorithms and Weight Design for Constrained Optimization

TL;DR

This paper introduces novel distributed approximate Newton algorithms with optimal weight design for constrained optimization, achieving faster convergence and scalability in economic dispatch and resource allocation problems.

Contribution

It develops a convex approximation for optimal edge weight design and proposes new distributed Newton algorithms with proven convergence for constrained optimization.

Findings

Algorithms converge linearly to the optimal solution.

Proposed methods outperform existing strategies in convergence speed.

Communication messages are of constant size, enabling scalability.

Abstract

Motivated by economic dispatch and linearly-constrained resource allocation problems, this paper proposes a class of novel Distributed-Approx Newton algorithms that approximate the standard Newton optimization method. We first develop the notion of an optimal edge weighting for the communication graph over which agents implement the second-order algorithm, and propose a convex approximation for the nonconvex weight design problem. We next build on the optimal weight design to develop a discrete Distributed Approx-Newton algorithm which converges linearly to the optimal solution for economic dispatch problems with unknown cost functions and relaxed local box constraints. For the full box-constrained problem, we develop a continuous Distributed Approx-Newton algorithm which is inspired by first-order saddle-point methods and rigorously prove its convergence to the primal and dual optimizers. A main property of each of these distributed algorithms is that they only require agents to exchange constant-size communication messages, which lends itself to scalable implementations. Simulations demonstrate that the Distributed Approx-Newton algorithms with our weight design have superior convergence properties compared to existing weighting strategies for first-order saddle-point and gradient descent methods.