Certificate-Guided Pruning for Stochastic Lipschitz Optimization

TL;DR

This paper introduces Certificate-Guided Pruning (CGP), an innovative method for black-box Lipschitz function optimization that provides explicit certificates of optimality, improves sample efficiency, and scales to high dimensions through several extensions.

Contribution

The paper presents CGP, a novel pruning method with explicit certificates, and extends it with adaptive Lipschitz learning, trust-region scaling, and hybrid refinement for high-dimensional problems.

Findings

CGP achieves near-optimal sample complexity bounds.

CGP variants outperform or match strong baselines on benchmarks.

Certificates enable principled stopping criteria and reliable optimization.

Abstract

We study black-box optimization of Lipschitz functions under noisy evaluations. Existing adaptive discretization methods implicitly avoid suboptimal regions but do not provide explicit certificates of optimality or measurable progress guarantees. We introduce \textbf{Certificate-Guided Pruning (CGP)}, which maintains an explicit \emph{active set} $A_t$ of potentially optimal points via confidence-adjusted Lipschitz envelopes. Any point outside $A_t$ is certifiably suboptimal with high probability, and under a margin condition with near-optimality dimension $\alpha$, we prove $\Vol(A_t)$ shrinks at a controlled rate yielding sample complexity $\tildeO(\varepsilon^{-(2+\alpha)})$. We develop three extensions: CGP-Adaptive learns $L$ online with $O(\log T)$ overhead; CGP-TR scales to $d > 50$ via trust regions with local certificates; and CGP-Hybrid switches to GP refinement when local smoothness is detected. Experiments on 12 benchmarks ($d \in [2, 100]$) show CGP variants match or exceed strong baselines while providing principled stopping criteria via certificate volume.