Regime-Adaptive Bayesian Optimization via Dirichlet Process Mixtures of Gaussian Processes

TL;DR

This paper introduces RAMBO, a Bayesian optimization method that adaptively models multiple regimes using Dirichlet Process Mixtures of Gaussian Processes, improving optimization in complex, multi-regime problems.

Contribution

The paper proposes RAMBO, a novel multi-regime Bayesian optimization framework with an efficient inference algorithm and adaptive regime discovery, addressing limitations of standard GPs.

Findings

Outperforms existing methods on synthetic benchmarks

Effective in molecular conformer optimization and drug discovery

Demonstrates robustness across diverse multi-regime applications

Abstract

Standard Bayesian Optimization (BO) assumes uniform smoothness across the search space an assumption violated in multi-regime problems such as molecular conformation search through distinct energy basins or drug discovery across heterogeneous molecular scaffolds. A single GP either oversmooths sharp transitions or hallucinates noise in smooth regions, yielding miscalibrated uncertainty. We propose RAMBO, a Dirichlet Process Mixture of Gaussian Processes that automatically discovers latent regimes during optimization, each modeled by an independent GP with locally-optimized hyperparameters. We derive collapsed Gibbs sampling that analytically marginalizes latent functions for efficient inference, and introduce adaptive concentration parameter scheduling for coarse-to-fine regime discovery. Our acquisition functions decompose uncertainty into intra-regime and inter-regime components. Experiments on synthetic benchmarks and real-world applications, including molecular conformer optimization, virtual screening for drug discovery, and fusion reactor design, demonstrate consistent improvements over state-of-the-art baselines on multi-regime objectives.