A Zeroth-Order Proximal Algorithm for Consensus Optimization

TL;DR

This paper introduces ZoPro, a zeroth-order proximal algorithm for distributed consensus optimization that approximates Hessian and gradient information, converging efficiently under certain conditions.

Contribution

The paper proposes ZoPro, a novel zeroth-order proximal algorithm integrating Hessian and gradient approximation for distributed consensus optimization.

Findings

ZoPro converges linearly to a neighborhood of the optimum.

ZoPro outperforms existing zeroth-order algorithms in convergence speed.

ZoPro surpasses some second-order algorithms in runtime efficiency.

Abstract

This paper considers a consensus optimization problem, where all the nodes in a network, with access to the zeroth-order information of its local objective function only, attempt to cooperatively achieve a common minimizer of the sum of their local objectives. To address this problem, we develop ZoPro, a zeroth-order proximal algorithm, which incorporates a zeroth-order oracle for approximating Hessian and gradient into a recently proposed, high-performance distributed second-order proximal algorithm. We show that the proposed ZoPro algorithm, equipped with a dynamic stepsize, converges linearly to a neighborhood of the optimum in expectation, provided that each local objective function is strongly convex and smooth. Extensive simulations demonstrate that ZoPro converges faster than several state-of-the-art distributed zeroth-order algorithms and outperforms a few distributed second-order algorithms in terms of running time for reaching given accuracy.