Continuous surrogate-based optimization algorithms are well-suited for expensive discrete problems

TL;DR

This paper demonstrates that continuous surrogate-based optimization methods are competitive with specialized discrete methods for high-dimensional, expensive discrete problems, challenging the common assumption that discrete-specific models are always superior.

Contribution

The paper provides empirical evidence that continuous surrogate models perform well on high-dimensional discrete problems, including real-world applications, contrary to prior beliefs.

Findings

Continuous surrogate models show competitive performance on discrete benchmarks.

Performance varies depending on problem type and time constraints.

Continuous methods can be effective alternatives to discrete-specific models.

Abstract

One method to solve expensive black-box optimization problems is to use a surrogate model that approximates the objective based on previous observed evaluations. The surrogate, which is cheaper to evaluate, is optimized instead to find an approximate solution to the original problem. In the case of discrete problems, recent research has revolved around surrogate models that are specifically constructed to deal with discrete structures. A main motivation is that literature considers continuous methods, such as Bayesian optimization with Gaussian processes as the surrogate, to be sub-optimal (especially in higher dimensions) because they ignore the discrete structure by, e.g., rounding off real-valued solutions to integers. However, we claim that this is not true. In fact, we present empirical evidence showing that the use of continuous surrogate models displays competitive performance on a set of high-dimensional discrete benchmark problems, including a real-life application, against state-of-the-art discrete surrogate-based methods. Our experiments on different discrete structures and time constraints also give more insight into which algorithms work well on which type of problem.