On Constructing Algorithm Portfolios in Algorithm Selection for Computationally Expensive Black-box Optimization in the Fixed-budget Setting

TL;DR

This paper emphasizes the importance of considering the number of function evaluations in constructing algorithm portfolios for fixed-budget black-box optimization, demonstrating improved performance over previous methods.

Contribution

It introduces a novel approach that accounts for evaluation budget in portfolio construction, addressing a gap in fixed-budget setting optimization.

Findings

Portfolios constructed with the new approach outperform previous methods.

Considering evaluation budget during construction improves optimizer selection.

The approach is effective for computationally expensive black-box optimization.

Abstract

Feature-based offline algorithm selection has shown its effectiveness in a wide range of optimization problems, including the black-box optimization problem. An algorithm selection system selects the most promising optimizer from an algorithm portfolio, which is a set of pre-defined optimizers. Thus, algorithm selection requires a well-constructed algorithm portfolio consisting of efficient optimizers complementary to each other. Although construction methods for the fixed-target setting have been well studied, those for the fixed-budget setting have received less attention. Here, the fixed-budget setting is generally used for computationally expensive optimization, where a budget of function evaluations is small. In this context, first, this paper points out some undesirable properties of experimental setups in previous studies. Then, this paper argues the importance of considering the number of function evaluations used in the sampling phase when constructing algorithm portfolios, whereas the previous studies ignored that. The results show that algorithm portfolios constructed by our approach perform significantly better than those by the previous approach.