Certified Multi-Fidelity Zeroth-Order Optimization

TL;DR

This paper introduces a certified multi-fidelity zeroth-order optimization algorithm that efficiently minimizes functions by adaptively balancing evaluation costs and providing data-driven error bounds, with theoretical guarantees.

Contribution

It formalizes the multi-fidelity zeroth-order optimization problem, proposes a certified variant of MFDOO, and establishes near-optimal cost complexity bounds with applications to noisy evaluations.

Findings

Proposed a certified MFDOO algorithm with bounded cost complexity.

Derived an $f$-dependent lower bound showing near-optimality.

Extended results to noisy evaluations and Lipschitz bandits.

Abstract

We consider the problem of multi-fidelity zeroth-order optimization, where one can evaluate a function $f$ at various approximation levels (of varying costs), and the goal is to optimize $f$ with the cheapest evaluations possible. In this paper, we study certified algorithms, which are additionally required to output a data-driven upper bound on the optimization error. We first formalize the problem in terms of a min-max game between an algorithm and an evaluation environment. We then propose a certified variant of the MFDOO algorithm and derive a bound on its cost complexity for any Lipschitz function $f$. We also prove an $f$-dependent lower bound showing that this algorithm has a near-optimal cost complexity. As a direct example, we close the paper by addressing the special case of noisy (stochastic) evaluations, which corresponds to $\eps$-best arm identification in Lipschitz bandits with continuously many arms.