Wasserstein Barycenter Gaussian Process based Bayesian Optimization

TL;DR

This paper introduces WBGP-BO, a novel Bayesian Optimization method that combines multiple Gaussian Processes into a Wasserstein barycenter, improving convergence especially on challenging problems.

Contribution

The paper proposes a new Bayesian Optimization approach using Wasserstein barycenters of Gaussian Processes, addressing hyperparameter tuning issues in traditional methods.

Findings

WBGP-BO converges to the optimum on difficult test problems.

The method outperforms vanilla Bayesian Optimization in challenging scenarios.

Wasserstein barycenters provide a robust alternative for Gaussian Process modeling.

Abstract

Gaussian Process based Bayesian Optimization is a widely applied algorithm to learn and optimize under uncertainty, well-known for its sample efficiency. However, recently -- and more frequently -- research studies have empirically demonstrated that the Gaussian Process fitting procedure at its core could be its most relevant weakness. Fitting a Gaussian Process means tuning its kernel's hyperparameters to a set of observations, but the common Maximum Likelihood Estimation technique, usually appropriate for learning tasks, has shown different criticalities in Bayesian Optimization, making theoretical analysis of this algorithm an open challenge. Exploiting the analogy between Gaussian Processes and Gaussian Distributions, we present a new approach which uses a prefixed set of hyperparameters values to fit as many Gaussian Processes and then combines them into a unique model as a Wasserstein Barycenter of Gaussian Processes. We considered both "easy" test problems and others known to undermine the \textit{vanilla} Bayesian Optimization algorithm. The new method, namely Wasserstein Barycenter Gausssian Process based Bayesian Optimization (WBGP-BO), resulted promising and able to converge to the optimum, contrary to vanilla Bayesian Optimization, also on the most "tricky" test problems.