Local Constrained Bayesian Optimization

TL;DR

This paper introduces Local Constrained Bayesian Optimization (LCBO), a new method designed for high-dimensional constrained problems that balances local descent and exploration, achieving better convergence and performance than existing approaches.

Contribution

LCBO is a novel framework that overcomes limitations of trust-region methods by using constraint-penalized surrogates and provides theoretical convergence guarantees.

Findings

LCBO achieves polynomial convergence rates for KKT residuals in high dimensions.

LCBO outperforms state-of-the-art methods on benchmarks up to 100 dimensions.

Theoretical analysis confirms LCBO's effectiveness under mild assumptions.

Abstract

Bayesian optimization (BO) for high-dimensional constrained problems remains a significant challenge due to the curse of dimensionality. We propose Local Constrained Bayesian Optimization (LCBO), a novel framework tailored for such settings. Unlike trust-region methods that are prone to premature shrinking when confronting tight or complex constraints, LCBO leverages the differentiable landscape of constraint-penalized surrogates to alternate between rapid local descent and uncertainty-driven exploration. Theoretically, we prove that LCBO achieves a convergence rate for the Karush-Kuhn-Tucker (KKT) residual that depends polynomially on the dimension $d$ for common kernels under mild assumptions, offering a rigorous alternative to global BO where regret bounds typically scale exponentially. Extensive evaluations on high-dimensional benchmarks (up to 100D) demonstrate that LCBO consistently outperforms state-of-the-art baselines.