Practical Bayesian Optimization of Objectives with Conditioning Variables

TL;DR

This paper introduces ConBO, a Bayesian optimization framework for problems with multiple objectives conditioned on a variable, leveraging data sharing and a new acquisition function to improve efficiency and performance.

Contribution

The paper proposes ConBO, a novel conditional Bayesian optimization framework that incorporates data sharing and a hybrid acquisition function for improved multi-objective optimization.

Findings

ConBO performs comparably or better than recent methods across various problems.

The hybrid Knowledge Gradient acquisition function enhances optimization efficiency.

The method is easily parallelizable for batch optimization.

Abstract

Bayesian optimization is a class of data efficient model based algorithms typically focused on global optimization. We consider the more general case where a user is faced with multiple problems that each need to be optimized conditional on a state variable, for example given a range of cities with different patient distributions, we optimize the ambulance locations conditioned on patient distribution. Given partitions of CIFAR-10, we optimize CNN hyperparameters for each partition. Similarity across objectives boosts optimization of each objective in two ways: in modelling by data sharing across objectives, and also in acquisition by quantifying how a single point on one objective can provide benefit to all objectives. For this we propose a framework for conditional optimization: ConBO. This can be built on top of a range of acquisition functions and we propose a new Hybrid Knowledge Gradient acquisition function. The resulting method is intuitive and theoretically grounded, performs either similar to or significantly better than recently published works on a range of problems, and is easily parallelized to collect a batch of points.