Bayesian Optimization in Linear Time

TL;DR

This paper introduces a novel Bayesian optimization method that uses recursive binary partitioning to achieve linear computational complexity and better local modeling, outperforming standard approaches on high-dimensional test functions.

Contribution

The authors propose a partitioning-based Bayesian optimization technique that addresses computational inefficiency and local modeling limitations of standard methods.

Findings

Achieves superior optimization performance on test functions with dimensions up to 124.

Maintains linear computational complexity, unlike traditional cubic complexity methods.

Outperforms a standard Bayesian optimization library across all tested functions.

Abstract

Bayesian optimization is a sequential method for minimizing objective functions that are expensive to evaluate and about which few assumptions can be made. By using all gathered data to train a Gaussian process model for the function and adaptively employing a mixture of global exploration and local exploitation, this method has been used for optimization in many fields including machine learning, automotive engineering and reinforcement learning. However, the standard method suffers from two problems: 1) with cubic computational complexity in the training-set size it eventually becomes computationally infeasible to train the model, and 2) globally modeling the objective function is not necessarily optimal given the local nature of minimization. Using flexible and recursive binary partitioning of the search space, we adapt both the modeling and acquisitive aspects of standard Bayesian optimization to work harmoniously with the partitioning scheme, thereby ameliorating both standard shortcomings. We compare our method against a commonly used Bayesian optimization library on seven challenging test functions, ranging in dimensionality from $6$ to $124$, and show that our method achieves superior optimization performance in all tests. In addition our method has linear computational complexity.