Randomized Kriging Believer for Parallel Bayesian Optimization with Regret Bounds

TL;DR

This paper introduces a new parallel Bayesian optimization method called randomized Kriging Believer, which offers theoretical regret guarantees and practical advantages like low complexity and versatility, improving optimization of expensive black-box functions.

Contribution

It proposes a novel randomized KB method for parallel Bayesian optimization that combines practical efficiency with theoretical regret bounds, addressing limitations of existing approaches.

Findings

Achieves Bayesian expected regret guarantees.

Demonstrates superior performance on synthetic and real-world benchmarks.

Offers low computational complexity and versatility across different BO settings.

Abstract

We consider an optimization problem of an expensive-to-evaluate black-box function, in which we can obtain noisy function values in parallel. For this problem, parallel Bayesian optimization (PBO) is a promising approach, which aims to optimize with fewer function evaluations by selecting a diverse input set for parallel evaluation. However, existing PBO methods suffer from poor practical performance or lack theoretical guarantees. In this study, we propose a PBO method, called randomized kriging believer (KB), based on a well-known KB heuristic and inheriting the advantages of the original KB: low computational complexity, a simple implementation, versatility across various BO methods, and applicability to asynchronous parallelization. Furthermore, we show that our randomized KB achieves Bayesian expected regret guarantees. We demonstrate the effectiveness of the proposed method through experiments on synthetic and benchmark functions and emulators of real-world data.