Rethinking Trust Region Bayesian Optimization in High Dimensions

TL;DR

This paper introduces AdaScale-TuRBO, a variant of Trust Region Bayesian Optimization that scales the Gaussian process lengthscale with problem dimension and trust region size, improving performance in high-dimensional optimization.

Contribution

The paper proposes a simple lengthscale scaling method for TuRBO to maintain kernel geometry and prior complexity, enhancing high-dimensional Bayesian optimization.

Findings

AdaScale-TuRBO outperforms standard TuRBO on benchmarks.

It maintains consistent prior complexity across dimensions.

It improves robustness in real-world trajectory planning.

Abstract

Trust Region Bayesian Optimization (TuRBO) is an effective strategy for alleviating the curse of dimensionality in high-dimensional black-box optimization. However, inappropriate lengthscale design can cause the local Gaussian process (GP) model within the trust region to degenerate, leading to suboptimal performance in high dimensions. In this work, we show that TuRBO's local GP may remain either excessively complex or overly simple as the dimension $D$ and trust region side length $L$ vary. To address this issue, we propose a straightforward variant, AdaScale-TuRBO, which scales the GP lengthscale with both the problem dimension and trust region size, thereby preserving kernel geometry and maintaining consistent prior complexity. Empirically, we show that AdaScale-TuRBO can robustly outperform standard TuRBO and other popular high-dimensional BO methods on synthetic benchmarks and real-world trajectory planning tasks.