Random Postprocessing for Combinatorial Bayesian Optimization

TL;DR

This paper introduces a postprocessing method for Bayesian optimization that prevents duplicate samples, significantly reducing the number of steps needed to find the global optimum in high-dimensional discrete problems.

Contribution

The paper proposes a simple, general postprocessing strategy that improves convergence speed of Bayesian optimization by avoiding repeated samples.

Findings

Reduces the number of steps to find the global optimum

Especially effective with maximum a posteriori acquisition functions

Enhances Bayesian optimization efficiency in high-dimensional spaces

Abstract

Model-based sequential approaches to discrete "black-box" optimization, including Bayesian optimization techniques, often access the same points multiple times for a given objective function in interest, resulting in many steps to find the global optimum. Here, we numerically study the effect of a postprocessing method on Bayesian optimization that strictly prohibits duplicated samples in the dataset. We find the postprocessing method significantly reduces the number of sequential steps to find the global optimum, especially when the acquisition function is of maximum a posterior estimation. Our results provide a simple but general strategy to solve the slow convergence of Bayesian optimization for high-dimensional problems.