The Importance of Landscape Features for Performance Prediction of Modular CMA-ES Variants

TL;DR

This paper investigates how landscape features, considering algorithm properties, can improve performance prediction of modular CMA-ES variants, showing that feature relevance varies with configurations and enabling module status prediction.

Contribution

It introduces a method to assess the impact of landscape features based on algorithm properties for better performance prediction of modular CMA-ES variants.

Findings

Most relevant features are configuration-independent.

Feature influence on regression accuracy varies with modules.

Classifiers can predict module status based on feature relevance.

Abstract

Selecting the most suitable algorithm and determining its hyperparameters for a given optimization problem is a challenging task. Accurately predicting how well a certain algorithm could solve the problem is hence desirable. Recent studies in single-objective numerical optimization show that supervised machine learning methods can predict algorithm performance using landscape features extracted from the problem instances. Existing approaches typically treat the algorithms as black-boxes, without consideration of their characteristics. To investigate in this work if a selection of landscape features that depends on algorithms properties could further improve regression accuracy, we regard the modular CMA-ES framework and estimate how much each landscape feature contributes to the best algorithm performance regression models. Exploratory data analysis performed on this data indicate that the set of most relevant features does not depend on the configuration of individual modules, but the influence that these features have on regression accuracy does. In addition, we have shown that by using classifiers that take the features relevance on the model accuracy, we are able to predict the status of individual modules in the CMA-ES configurations.