Surrogate-based categorical neighborhoods for mixed-variable blackbox optimization

TL;DR

This paper introduces a surrogate-based approach to handle categorical variables in constrained mixed-variable blackbox optimization, improving neighborhood construction and optimization performance.

Contribution

It proposes a systematic method to model and structure categorical variables using surrogate models, enabling automatic, constraint-aware neighborhood construction in mixed-variable optimization.

Findings

CatMADS-GP outperforms state-of-the-art solvers on benchmark problems.

Surrogate-based neighborhoods improve handling of categorical variables.

Method is effective for both constrained and unconstrained problems.

Abstract

In simulation-based engineering, design choices are often obtained following the optimization of complex blackbox models. These models frequently involve mixed-variable domains with quantitative and categorical variables. Unlike quantitative variables, categorical variables lack an inherent structure, which makes them difficult to handle, especially in the presence of constraints. This work proposes a systematic approach to structure and model categorical variables in constrained mixed-variable blackbox optimization. Surrogate models of the objective and constraint functions are used to induce problem-specific categorical distances. From these distances, surrogate-based neighborhoods are constructed using notions of dominance from bi-objective optimization, jointly accounting for information from both the objective and the constraint functions. This study addresses the lack of automatic and constraint-aware categorical neighborhood construction in mixed-variable blackbox optimization. As a proof of concept, these neighborhoods are employed within CatMADS, an extension of the MADS algorithm for categorical variables. The surrogate models are Gaussian processes, and the resulting method is called CatMADS-GP. The method is benchmarked on the Cat-Suite collection of 60 mixed-variable optimization problems and compared against state-of-the-art solvers. Data profiles indicate that CatMADS-GP achieves superior performance for both unconstrained and constrained problems.