New Heuristics for Parallel and Scalable Bayesian Optimization

TL;DR

This paper reviews current parallel Bayesian optimization methods, identifies common pitfalls, and introduces practical heuristics and open-source tools to enhance scalability and ease of deployment in various computing environments.

Contribution

It offers new heuristic algorithms and software implementations to improve the scalability and robustness of Bayesian optimization in parallel computing settings.

Findings

Heuristic algorithms effectively mitigate oversampling and over-exploitation issues.

Open source implementations facilitate easy deployment across computing environments.

Practical guidelines improve Bayesian optimization performance in parallel settings.

Abstract

Bayesian optimization has emerged as a strong candidate tool for global optimization of functions with expensive evaluation costs. However, due to the dynamic nature of research in Bayesian approaches, and the evolution of computing technology, using Bayesian optimization in a parallel computing environment remains a challenge for the non-expert. In this report, I review the state-of-the-art in parallel and scalable Bayesian optimization methods. In addition, I propose practical ways to avoid a few of the pitfalls of Bayesian optimization, such as oversampling of edge parameters and over-exploitation of high performance parameters. Finally, I provide relatively simple, heuristic algorithms, along with their open source software implementations, that can be immediately and easily deployed in any computing environment.