Linear Embedding-based High-dimensional Batch Bayesian Optimization without Reconstruction Mappings

TL;DR

This paper proposes a high-dimensional Bayesian optimization method that directly operates in the original space using learned low-dimensional structures, avoiding reconstruction biases and enabling efficient batch optimization in thousands of dimensions.

Contribution

It introduces a novel approach that bypasses reconstruction mappings, allowing effective high-dimensional optimization directly in the original space with theoretical exploration guarantees.

Findings

Effective in high-dimensional benchmarks

Applicable to real-world functions with thousands of dimensions

Outperforms existing methods in exploration efficiency

Abstract

The optimization of high-dimensional black-box functions is a challenging problem. When a low-dimensional linear embedding structure can be assumed, existing Bayesian optimization (BO) methods often transform the original problem into optimization in a low-dimensional space. They exploit the low-dimensional structure and reduce the computational burden. However, we reveal that this approach could be limited or inefficient in exploring the high-dimensional space mainly due to the biased reconstruction of the high-dimensional queries from the low-dimensional queries. In this paper, we investigate a simple alternative approach: tackling the problem in the original high-dimensional space using the information from the learned low-dimensional structure. We provide a theoretical analysis of the exploration ability. Furthermore, we show that our method is applicable to batch optimization problems with thousands of dimensions without any computational difficulty. We demonstrate the effectiveness of our method on high-dimensional benchmarks and a real-world function.