High dimensional Bayesian Optimization via Condensing-Expansion Projection

TL;DR

This paper introduces CEPBO, a new high-dimensional Bayesian optimization method using random projections that does not rely on the effective subspace assumption, outperforming existing algorithms.

Contribution

The paper proposes CEPBO, a novel random projection-based Bayesian optimization approach that is simple, practical, and effective without the effective subspace assumption.

Findings

CEPBO outperforms existing random embedding algorithms in most high-dimensional BO tasks.

Both Gaussian and hashing projection matrices are effective in CEPBO.

Experimental results validate the superiority of CEPBO in high-dimensional settings.

Abstract

In high-dimensional settings, Bayesian optimization (BO) can be expensive and infeasible. The random embedding Bayesian optimization algorithm is commonly used to address high-dimensional BO challenges. However, this method relies on the effective subspace assumption on the optimization problem's objective function, which limits its applicability. In this paper, we introduce Condensing-Expansion Projection Bayesian optimization (CEPBO), a novel random projection-based approach for high-dimensional BO that does not reply on the effective subspace assumption. The approach is both simple to implement and highly practical. We present two algorithms based on different random projection matrices: the Gaussian projection matrix and the hashing projection matrix. Experimental results demonstrate that both algorithms outperform existing random embedding-based algorithms in most cases, achieving superior performance on high-dimensional BO problems. The code is available in \url{https://anonymous.4open.science/r/CEPBO-14429}.