Black-box Mixed-Variable Optimisation using a Surrogate Model that Satisfies Integer Constraints

TL;DR

This paper introduces MVRSM, a surrogate-based optimization algorithm that effectively handles mixed continuous and integer variables, outperforming existing methods on benchmarks and real-world problems.

Contribution

The paper presents MVRSM, a novel surrogate model that ensures integer constraints are satisfied at local optima, advancing mixed-variable optimization techniques.

Findings

Outperforms state-of-the-art on synthetic benchmarks with up to 238 variables

Achieves competitive results on XGBoost hyperparameter tuning

Effective in electrostatic precipitator optimization

Abstract

A challenging problem in both engineering and computer science is that of minimising a function for which we have no mathematical formulation available, that is expensive to evaluate, and that contains continuous and integer variables, for example in automatic algorithm configuration. Surrogate-based algorithms are very suitable for this type of problem, but most existing techniques are designed with only continuous or only discrete variables in mind. Mixed-Variable ReLU-based Surrogate Modelling (MVRSM) is a surrogate-based algorithm that uses a linear combination of rectified linear units, defined in such a way that (local) optima satisfy the integer constraints. This method outperforms the state of the art on several synthetic benchmarks with up to 238 continuous and integer variables, and achieves competitive performance on two real-life benchmarks: XGBoost hyperparameter tuning and Electrostatic Precipitator optimisation.