A warped kernel improving robustness in Bayesian optimization via random embeddings

TL;DR

This paper introduces a warped kernel for Bayesian optimization that enhances robustness and efficiency in high-dimensional spaces by mitigating extrinsic dimensionality issues and reducing redundant evaluations.

Contribution

It proposes a novel kernel warping method within the random embedding Bayesian optimization framework, improving robustness and flexibility in high-dimensional optimization tasks.

Findings

Improved robustness in high-dimensional Bayesian optimization.

Reduced redundant evaluations through kernel warping.

Enhanced handling of bound constraints in embedded domains.

Abstract

This works extends the Random Embedding Bayesian Optimization approach by integrating a warping of the high dimensional subspace within the covariance kernel. The proposed warping, that relies on elementary geometric considerations, allows mitigating the drawbacks of the high extrinsic dimensionality while avoiding the algorithm to evaluate points giving redundant information. It also alleviates constraints on bound selection for the embedded domain, thus improving the robustness, as illustrated with a test case with 25 variables and intrinsic dimension 6.