Comparing Algorithm Selection Approaches on Black-Box Optimization Problems

TL;DR

This paper compares different machine learning models for algorithm selection in black-box optimization, showing that the choice of ML technique has minor impact on performance across various models and budgets.

Contribution

It provides an empirical comparison of four ML models for algorithm selection, highlighting the limited influence of the specific ML technique used.

Findings

ML model choice has minor impact on selection performance

Tree-based models perform consistently well

Per-instance algorithm selection shows strong potential

Abstract

Performance complementarity of solvers available to tackle black-box optimization problems gives rise to the important task of algorithm selection (AS). Automated AS approaches can help replace tedious and labor-intensive manual selection, and have already shown promising performance in various optimization domains. Automated AS relies on machine learning (ML) techniques to recommend the best algorithm given the information about the problem instance. Unfortunately, there are no clear guidelines for choosing the most appropriate one from a variety of ML techniques. Tree-based models such as Random Forest or XGBoost have consistently demonstrated outstanding performance for automated AS. Transformers and other tabular deep learning models have also been increasingly applied in this context. We investigate in this work the impact of the choice of the ML technique on AS performance. We compare four ML models on the task of predicting the best solver for the BBOB problems for 7 different runtime budgets in 2 dimensions. While our results confirm that a per-instance AS has indeed impressive potential, we also show that the particular choice of the ML technique is of much minor importance.