New Paradigms for Exploiting Parallel Experiments in Bayesian Optimization

TL;DR

This paper introduces new parallel Bayesian optimization methods that leverage system structure to efficiently explore the design space, significantly reducing search time and improving the likelihood of global solutions.

Contribution

The paper proposes novel parallel BO paradigms based on space partitioning by level sets and exploiting partial separability, enhancing parallelization and efficiency.

Findings

Significantly reduced search time in experiments.

Increased probability of finding global solutions.

Outperformed existing parallel algorithms in benchmarks.

Abstract

Bayesian optimization (BO) is one of the most effective methods for closed-loop experimental design and black-box optimization. However, a key limitation of BO is that it is an inherently sequential algorithm (one experiment is proposed per round) and thus cannot directly exploit high-throughput (parallel) experiments. Diverse modifications to the BO framework have been proposed in the literature to enable exploitation of parallel experiments but such approaches are limited in the degree of parallelization that they can achieve and can lead to redundant experiments (thus wasting resources and potentially compromising performance). In this work, we present new parallel BO paradigms that exploit the structure of the system to partition the design space. Specifically, we propose an approach that partitions the design space by following the level sets of the performance function and an approach that exploits partially-separable structures of the performance function found. We conduct extensive numerical experiments using a reactor case study to benchmark the effectiveness of these approaches against a variety of state-of-the-art parallel algorithms reported in the literature. Our computational results show that our approaches significantly reduce the required search time and increase the probability of finding a global (rather than local) solution.