High-Dimensional Bayesian Optimization Using Both Random and Supervised Embeddings

TL;DR

EGORSE is a novel high-dimensional Bayesian optimization method that adaptively combines random and supervised embeddings to efficiently optimize complex blackbox functions with fewer evaluations.

Contribution

This paper introduces EGORSE, a new Bayesian optimization approach that adaptively integrates random and supervised linear embeddings for high-dimensional problems.

Findings

EGORSE outperforms state-of-the-art algorithms in high-dimensional settings.

EGORSE reduces CPU time and blackbox calls significantly.

Effective for problems with up to 600 variables.

Abstract

Bayesian optimization (BO) is one of the most powerful strategies to solve computationally expensive-to-evaluate blackbox optimization problems. However, BO methods are conventionally used for optimization problems of small dimension because of the curse of dimensionality. In this paper, a high-dimensionnal optimization method incorporating linear embedding subspaces of small dimension is proposed to efficiently perform the optimization. An adaptive learning strategy for these linear embeddings is carried out in conjunction with the optimization. The resulting BO method, named efficient global optimization coupled with random and supervised embedding (EGORSE), combines in an adaptive way both random and supervised linear embeddings. EGORSE has been compared to state-of-the-art algorithms and tested on academic examples with a number of design variables ranging from 10 to 600. The obtained results show the high potential of EGORSE to solve high-dimensional blackbox optimization problems, in terms of both CPU time and the limited number of calls to the expensive blackbox simulation.