The Impact of Hyper-Parameter Tuning for Landscape-Aware Performance Regression and Algorithm Selection

TL;DR

This paper investigates how hyper-parameter tuning of classical regression models like random forests and decision trees significantly influences the effectiveness of landscape-aware algorithm selection in evolutionary computation.

Contribution

It systematically evaluates the impact of hyper-parameters on regression models used in ELA-based algorithm selection, providing guidelines for better tuning practices.

Findings

Hyper-parameter tuning greatly affects regression model quality.

Systematic tuning improves algorithm selector performance.

Guidelines for tuning classical ML models in ELA are proposed.

Abstract

Automated algorithm selection and configuration methods that build on exploratory landscape analysis (ELA) are becoming very popular in Evolutionary Computation. However, despite a significantly growing number of applications, the underlying machine learning models are often chosen in an ad-hoc manner. We show in this work that three classical regression methods are able to achieve meaningful results for ELA-based algorithm selection. For those three models -- random forests, decision trees, and bagging decision trees -- the quality of the regression models is highly impacted by the chosen hyper-parameters. This has significant effects also on the quality of the algorithm selectors that are built on top of these regressions. By comparing a total number of 30 different models, each coupled with 2 complementary regression strategies, we derive guidelines for the tuning of the regression models and provide general recommendations for a more systematic use of classical machine learning models in landscape-aware algorithm selection. We point out that a choice of the machine learning model merits to be carefully undertaken and further investigated.