A Proximal Zeroth-Order Algorithm for Nonconvex Nonsmooth Problems

TL;DR

This paper introduces a proximal zeroth-order primal dual algorithm for nonconvex nonsmooth problems, enabling optimization using only function evaluations, suitable for noisy settings, with proven convergence and validated by experiments.

Contribution

It presents a novel zeroth-order algorithm tailored for nonconvex nonsmooth problems with convergence guarantees, addressing scenarios with only noisy function evaluations.

Findings

Proposed algorithm converges under certain conditions.

Achieved convergence rates for the zeroth-order method.

Numerical experiments confirm theoretical results.

Abstract

In this paper, we focus on solving an important class of nonconvex optimization problems which includes many problems for example signal processing over a networked multi-agent system and distributed learning over networks. Motivated by many applications in which the local objective function is the sum of smooth but possibly nonconvex part, and non-smooth but convex part subject to a linear equality constraint, this paper proposes a proximal zeroth-order primal dual algorithm (PZO-PDA) that accounts for the information structure of the problem. This algorithm only utilize the zeroth-order information (i.e., the functional values) of smooth functions, yet the flexibility is achieved for applications that only noisy information of the objective function is accessible, where classical methods cannot be applied. We prove convergence and rate of convergence for PZO-PDA. Numerical experiments are provided to validate the theoretical results.