Thinking inside the box: A tutorial on grey-box Bayesian optimization

TL;DR

This paper provides a comprehensive tutorial on grey-box Bayesian optimization, which leverages internal information about objective functions to improve optimization efficiency, especially in composite and multi-fidelity settings.

Contribution

It introduces and explains grey-box BO methods, combining black-box and white-box approaches, with a focus on composite objectives and multi-fidelity optimization techniques.

Findings

Grey-box BO significantly improves optimization performance.

Methods enable selective evaluation of objective components.

Tutorial clarifies application of grey-box techniques in practice.

Abstract

Bayesian optimization (BO) is a framework for global optimization of expensive-to-evaluate objective functions. Classical BO methods assume that the objective function is a black box. However, internal information about objective function computation is often available. For example, when optimizing a manufacturing line's throughput with simulation, we observe the number of parts waiting at each workstation, in addition to the overall throughput. Recent BO methods leverage such internal information to dramatically improve performance. We call these "grey-box" BO methods because they treat objective computation as partially observable and even modifiable, blending the black-box approach with so-called "white-box" first-principles knowledge of objective function computation. This tutorial describes these methods, focusing on BO of composite objective functions, where one can observe and selectively evaluate individual constituents that feed into the overall objective; and multi-fidelity BO, where one can evaluate cheaper approximations of the objective function by varying parameters of the evaluation oracle.