ORTHOBO: Orthogonal Bayesian Hyperparameter Optimization

TL;DR

OrthoBO introduces an orthogonal variance reduction technique for Bayesian hyperparameter optimization, improving stability and performance in noisy evaluation scenarios.

Contribution

It proposes an orthogonal acquisition estimator and a comprehensive Bayesian optimization framework that enhances stability and efficiency.

Findings

Reduces Monte Carlo variance in acquisition estimates.

Stabilizes candidate rankings in Bayesian optimization.

Achieves improved hyperparameter tuning performance.

Abstract

Bayesian optimization is widely used for hyperparameter optimization when model evaluations are expensive; however, noisy acquisition estimates can lead to unstable decisions. We identify acquisition estimation noise as a failure mode that was previously overlooked: even when the surrogate model and acquisition target are correctly specified, finite-sample Monte Carlo error can perturb acquisition values. This can, in turn, flip candidate rankings and lead to suboptimal BO decisions. As a remedy, we aim at variance reduction and propose an orthogonal acquisition estimator that subtracts an optimally weighted score-function control variate, which yields an acquisition residual orthogonal to posterior score directions and which thus reduces Monte Carlo variance. We further introduce OrthoBO: a Bayesian optimization framework that combines our orthogonal acquisition estimator with ensemble surrogates and an outer log transformation. We show theoretically that our estimator preserves the target, leads to variance reduction, and improves pairwise ranking stability. We further verify the theoretical properties of OrthoBO through numerical experiments where our framework reduces acquisition estimation variance, stabilizes candidate rankings, and achieves strong performance. We also demonstrate the downstream utility of OrthoBO in hyperparameter optimization for neural network training and fine-tuning.