Safe Zeroth-Order Convex Optimization Using Quadratic Local Approximations

TL;DR

This paper introduces SZO-QQ, a zeroth-order convex optimization method that uses quadratic local approximations to efficiently find feasible points and converge to the optimal solution in black-box settings.

Contribution

The paper presents a novel zeroth-order optimization algorithm leveraging quadratic approximations, improving convergence speed in black-box convex optimization problems.

Findings

Faster convergence compared to existing zeroth-order methods

Proven convergence of objective values to the minimum

Effective in black-box convex optimization scenarios

Abstract

We address black-box convex optimization problems, where the objective and constraint functions are not explicitly known but can be sampled within the feasible set. The challenge is thus to generate a sequence of feasible points converging towards an optimal solution. By leveraging the knowledge of the smoothness properties of the objective and constraint functions, we propose a novel zeroth-order method, SZO-QQ, that iteratively computes quadratic approximations of the constraint functions, constructs local feasible sets and optimizes over them. We prove convergence of the sequence of the objective values generated at each iteration to the minimum. Through experiments, we show that our method can achieve faster convergence compared with state-of-the-art zeroth-order approaches to convex optimization.