Wasserstein-enabled characterization of designs and myopic decisions in Bayesian Optimization

TL;DR

This paper introduces a Wasserstein-based method to characterize and evaluate initial designs in Bayesian Optimization, aiming to improve decision-making and inspire new acquisition functions.

Contribution

It proposes a novel, model-free characterization of designs using Wasserstein distance, linking design properties to query quality in Bayesian Optimization.

Findings

Wasserstein-based measures effectively evaluate design quality.

Design characterization correlates with the success of subsequent queries.

Insights may lead to new acquisition function development.

Abstract

Impractical assumptions, an inherently myopic nature, and the crucial role of the initial design, all together contribute to making theoretical convergence proofs of little value in real-life Bayesian Optimization applications. In this paper, we propose a novel characterization of the design depending on its distributional properties, separately measured with respect to the coverage of the search space and the concentration around the best observed function value. These measures are based on the Wasserstein distance and enable a model-free evaluation of the information value of the design before deciding the next query. Then, embracing the myopic nature of Bayesian Optimization, we take an empirical approach to analyze the relation between the proposed characterization of the design and the quality of the next query. Ultimately, we provide important and useful insights that might inspire the definition of a new generation of acquisition functions in Bayesian Optimization.