Similarity-based Portfolio Construction for Black-box Optimization

TL;DR

This paper introduces a similarity-based portfolio construction method for black-box optimization, improving algorithm selection and performance on unseen problems by leveraging instance similarity and k-nearest neighbors.

Contribution

It proposes a simple k-nearest-neighbor-based finetuning approach for portfolio construction tailored to unseen instances, enhancing black-box optimization performance.

Findings

Naive portfolio over the full training set outperforms the virtual best solver.

k-nearest-neighbor finetuning further improves portfolio performance.

Portfolio methods outperform traditional single-algorithm baselines.

Abstract

In black-box optimization, a central question is which algorithm to use to solve a given, previously unseen, problem. Selecting a single algorithm, however, entails inherent risks: inaccuracies in the selector may lead to poor choices, and even well-performing algorithms with high variance can yield unsatisfactory results in a single run. A natural remedy is to split the evaluation budget across multiple runs of potentially different algorithms. Such sequential algorithm portfolios benefit from variance reduction and complementarities between algorithms, often outperforming approaches that allocate the entire budget to a single solver. While effective portfolios can be constructed post-hoc, transferring this idea to the algorithm selection setting is non-trivial. We show that a naive portfolio constructed over the full training set already outperforms the strongest traditional baseline, the virtual best solver. We then propose a simple yet effective k-nearest-neighbor-based finetuning approach to construct portfolios tailored to unseen instances, yielding further improvements and highlighting the effectiveness of portfolio selection in fixed-budget black-box optimization.