Bayesian Optimization for Mixed-Variable Problems in the Natural Sciences

TL;DR

This paper advances Bayesian optimization techniques for mixed-variable scientific problems by generalizing probabilistic reparameterization, enabling efficient gradient-based optimization in complex, real-world scenarios.

Contribution

It introduces a generalized PR approach for non-equidistant discrete variables, improving BO's effectiveness in fully mixed-variable settings with Gaussian process surrogates.

Findings

Demonstrates robustness of the generalized PR method on synthetic and experimental tasks.

Shows improved optimization of highly discontinuous and discretized objectives.

Establishes a practical BO framework for natural sciences with noisy, limited data.

Abstract

Optimizing expensive black-box objectives over mixed search spaces is a common challenge across the natural sciences. Bayesian optimization (BO) offers sample-efficient strategies through probabilistic surrogate models and acquisition functions. However, its effectiveness diminishes in mixed or high-cardinality discrete spaces, where gradients are unavailable and optimizing the acquisition function becomes computationally demanding. In this work, we generalize the probabilistic reparameterization (PR) approach of Daulton et al. to handle non-equidistant discrete variables, enabling gradient-based optimization in fully mixed-variable settings with Gaussian process (GP) surrogates. With real-world scientific optimization tasks in mind, we conduct systematic benchmarks on synthetic and experimental objectives to obtain an optimized kernel formulations and demonstrate the robustness of our generalized PR method. We additionally show that, when combined with a modified BO workflow, our approach can efficiently optimize highly discontinuous and discretized objective landscapes. This work establishes a practical BO framework for addressing fully mixed optimization problems in the natural sciences, and is particularly well suited to autonomous laboratory settings where noise, discretization, and limited data are inherent.