Warm Starting Bayesian Optimization

TL;DR

This paper introduces a framework for warm-starting Bayesian optimization by building a joint model of related problems, significantly reducing solution times in sequential decision-making scenarios involving stochastic simulators.

Contribution

It proposes a novel method to warm start Bayesian optimization across related problems by modeling their objectives jointly, enabling faster convergence.

Findings

Reduces optimization time in sequential problem settings

Effective for stochastic simulators without derivative information

Improves decision-making efficiency over traditional methods

Abstract

We develop a framework for warm-starting Bayesian optimization, that reduces the solution time required to solve an optimization problem that is one in a sequence of related problems. This is useful when optimizing the output of a stochastic simulator that fails to provide derivative information, for which Bayesian optimization methods are well-suited. Solving sequences of related optimization problems arises when making several business decisions using one optimization model and input data collected over different time periods or markets. While many gradient-based methods can be warm started by initiating optimization at the solution to the previous problem, this warm start approach does not apply to Bayesian optimization methods, which carry a full metamodel of the objective function from iteration to iteration. Our approach builds a joint statistical model of the entire collection of related objective functions, and uses a value of information calculation to recommend points to evaluate.