Boundary Exploration for Bayesian Optimization With Unknown Physical Constraints

TL;DR

This paper introduces BE-CBO, a Bayesian optimization approach that effectively explores the boundary between feasible and infeasible regions in problems with unknown constraints, outperforming existing methods.

Contribution

The paper proposes a novel boundary exploration technique for Bayesian optimization with unknown constraints, using neural network ensembles to better identify complex feasibility boundaries.

Findings

BE-CBO outperforms state-of-the-art methods on synthetic benchmarks.

Neural network ensembles better capture complex constraint boundaries.

The method is effective on real-world optimization problems.

Abstract

Bayesian optimization has been successfully applied to optimize black-box functions where the number of evaluations is severely limited. However, in many real-world applications, it is hard or impossible to know in advance which designs are feasible due to some physical or system limitations. These issues lead to an even more challenging problem of optimizing an unknown function with unknown constraints. In this paper, we observe that in such scenarios optimal solution typically lies on the boundary between feasible and infeasible regions of the design space, making it considerably more difficult than that with interior optima. Inspired by this observation, we propose BE-CBO, a new Bayesian optimization method that efficiently explores the boundary between feasible and infeasible designs. To identify the boundary, we learn the constraints with an ensemble of neural networks that outperform the standard Gaussian Processes for capturing complex boundaries. Our method demonstrates superior performance against state-of-the-art methods through comprehensive experiments on synthetic and real-world benchmarks. Code available at: https://github.com/yunshengtian/BE-CBO