SCORE: A 1D Reparameterization Technique to Break Bayesian Optimization's Curse of Dimensionality

TL;DR

SCORE introduces a 1D reparameterization method that significantly reduces the computational complexity of Bayesian optimization in high-dimensional spaces, enabling scalable and efficient global optimization.

Contribution

The paper proposes a novel 1D reparameterization technique that breaks the curse of dimensionality in Bayesian optimization, maintaining linear time complexity.

Findings

Successfully finds global minima in high-dimensional problems

Fits real-world data without high-performance computing resources

Maintains linear time complexity in high-dimensional landscapes

Abstract

Bayesian optimization (BO) has emerged as a powerful tool for navigating complex search spaces, showcasing practical applications in the fields of science and engineering.However, since it typically relies on a surrogate model to approximate the objective function, BO grapples with heightened computational costs that tend to escalate as the number of parameters and experiments grows. Several methods such as parallelization, surrogate model approximations, and memory pruning have been proposed to cut down computing time, but they all fall short of resolving the core issue behind BO's curse of dimensionality. In this paper, a 1D reparametrization trick is proposed to break this curse and sustain linear time complexity for BO in high-dimensional landscapes. This fast and scalable approach named SCORE can successfully find the global minimum of needle-in-a-haystack optimization functions and fit real-world data without the high-performance computing resources typically required by state-of-the-art techniques.