Navigating in High-Dimensional Search Space: A Hierarchical Bayesian Optimization Approach

TL;DR

This paper introduces HiBO, a hierarchical Bayesian optimization method that effectively navigates high-dimensional search spaces by combining global partitioning with local search, outperforming existing methods in benchmarks and real-world DBMS tuning.

Contribution

The paper proposes a novel hierarchical algorithm that integrates global search space partitioning into local Bayesian optimization, improving high-dimensional optimization performance.

Findings

HiBO outperforms state-of-the-art methods in synthetic benchmarks.

HiBO demonstrates significant practical effectiveness in tuning DBMS configurations.

The hierarchical approach effectively guides local search using global partitioning information.

Abstract

Optimizing black-box functions in high-dimensional search spaces has been known to be challenging for traditional Bayesian Optimization (BO). In this paper, we introduce HiBO, a novel hierarchical algorithm integrating global-level search space partitioning information into the acquisition strategy of a local BO-based optimizer. HiBO employs a search-tree-based global-level navigator to adaptively split the search space into partitions with different sampling potential. The local optimizer then utilizes this global-level information to guide its acquisition strategy towards most promising regions within the search space. A comprehensive set of evaluations demonstrates that HiBO outperforms state-of-the-art methods in high-dimensional synthetic benchmarks and presents significant practical effectiveness in the real-world task of tuning configurations of database management systems (DBMSs).