A Unified Zeroth-Order Proximal Newton-Type Framework for Composite Optimization

TL;DR

This paper introduces a unified derivative-free proximal Newton framework for composite optimization, providing complexity bounds, convergence analysis, and demonstrating efficiency through numerical experiments.

Contribution

It develops a novel unified zeroth-order proximal Newton-type algorithm with theoretical guarantees and practical efficiency for composite optimization problems.

Findings

Established iteration and oracle complexity bounds for nonconvex and strongly convex cases.

Proved local R-superlinear convergence under Dennis–Moré condition.

Numerical experiments show the method's efficiency in practice.

Abstract

We propose a unified derivative-free proximal Newton-type algorithm framework for solving composite optimization problems formulated as the sum of a black-box function and a known regularization term. We establish the iteration and oracle complexity bounds for the algorithm to attain an $\epsilon$-optimal solution under both nonconvex and strongly convex settings. We also establish its local R-superlinear convergence based on the Dennis--Mor\'{e} condition, and theoretically address an open problem by showing that the BFGS scheme is more compatible with finite-difference gradient estimators than with smoothing-based ones. Numerical experiments are further presented to demonstrate the efficiency of the proposed method.