Heteroscedastic Treed Bayesian Optimisation

TL;DR

This paper introduces a heteroscedastic treed Bayesian optimisation method that effectively models non-stationary functions, leading to improved performance in applications like machine learning and mining exploration.

Contribution

It proposes a novel hierarchical prior model with parameter learning to address non-stationarity in Bayesian optimisation, enhancing its flexibility and accuracy.

Findings

Significant performance improvements in diverse applications

Better modeling of non-stationary functions

Enhanced Bayesian optimisation accuracy

Abstract

Optimising black-box functions is important in many disciplines, such as tuning machine learning models, robotics, finance and mining exploration. Bayesian optimisation is a state-of-the-art technique for the global optimisation of black-box functions which are expensive to evaluate. At the core of this approach is a Gaussian process prior that captures our belief about the distribution over functions. However, in many cases a single Gaussian process is not flexible enough to capture non-stationarity in the objective function. Consequently, heteroscedasticity negatively affects performance of traditional Bayesian methods. In this paper, we propose a novel prior model with hierarchical parameter learning that tackles the problem of non-stationarity in Bayesian optimisation. Our results demonstrate substantial improvements in a wide range of applications, including automatic machine learning and mining exploration.