Bounce: Reliable High-Dimensional Bayesian Optimization for Combinatorial and Mixed Spaces

TL;DR

Bounce introduces a reliable Bayesian optimization method for high-dimensional, mixed, and combinatorial spaces, addressing the limitations of existing approaches by using nested embeddings to improve robustness and performance.

Contribution

The paper proposes Bounce, a novel Bayesian optimization algorithm that maps variable types into nested embeddings, enhancing reliability in high-dimensional combinatorial and mixed spaces.

Findings

Bounce reliably outperforms existing methods on various high-dimensional problems.

Bounce often improves upon state-of-the-art performance.

The method maintains robustness even when optima lack specific structure.

Abstract

Impactful applications such as materials discovery, hardware design, neural architecture search, or portfolio optimization require optimizing high-dimensional black-box functions with mixed and combinatorial input spaces. While Bayesian optimization has recently made significant progress in solving such problems, an in-depth analysis reveals that the current state-of-the-art methods are not reliable. Their performances degrade substantially when the unknown optima of the function do not have a certain structure. To fill the need for a reliable algorithm for combinatorial and mixed spaces, this paper proposes Bounce that relies on a novel map of various variable types into nested embeddings of increasing dimensionality. Comprehensive experiments show that Bounce reliably achieves and often even improves upon state-of-the-art performance on a variety of high-dimensional problems.