Which Surrogate Works for Empirical Performance Modelling? A Case Study with Differential Evolution

TL;DR

This paper investigates the effectiveness of different surrogate modeling techniques for predicting the empirical performance of differential evolution algorithms, aiming to improve algorithm configuration and understanding.

Contribution

It compares four regression algorithms as surrogates for modeling differential evolution performance, providing insights into their predictive accuracy and landscape approximation capabilities.

Findings

Regression models vary in predictive accuracy

Some surrogates better approximate performance landscapes

Insights aid in automatic algorithm configuration

Abstract

It is not uncommon that meta-heuristic algorithms contain some intrinsic parameters, the optimal configuration of which is crucial for achieving their peak performance. However, evaluating the effectiveness of a configuration is expensive, as it involves many costly runs of the target algorithm. Perhaps surprisingly, it is possible to build a cheap-to-evaluate surrogate that models the algorithm's empirical performance as a function of its parameters. Such surrogates constitute an important building block for understanding algorithm performance, algorithm portfolio/selection, and the automatic algorithm configuration. In principle, many off-the-shelf machine learning techniques can be used to build surrogates. In this paper, we take the differential evolution (DE) as the baseline algorithm for proof-of-concept study. Regression models are trained to model the DE's empirical performance given a parameter configuration. In particular, we evaluate and compare four popular regression algorithms both in terms of how well they predict the empirical performance with respect to a particular parameter configuration, and also how well they approximate the parameter versus the empirical performance landscapes.