Linear Matrix Factorization Embeddings for Single-objective Optimization Landscapes

TL;DR

This paper introduces a linear matrix factorization approach to preprocess landscape features in black-box optimization, improving correlation detection and aiding automated algorithm design.

Contribution

It proposes a novel linear matrix factorization method for feature preprocessing, enhancing interpretability and correlation detection in optimization landscapes.

Findings

Linear matrix factorization improves feature correlation detection.

Preprocessing with this method enhances automated algorithm selection.

The approach aids in better understanding of problem landscape characteristics.

Abstract

Automated per-instance algorithm selection and configuration have shown promising performances for a number of classic optimization problems, including satisfiability, AI planning, and TSP. The techniques often rely on a set of features that measure some characteristics of the problem instance at hand. In the context of black-box optimization, these features have to be derived from a set of $(x,f(x))$ samples. A number of different features have been proposed in the literature, measuring, for example, the modality, the separability, or the ruggedness of the instance at hand. Several of the commonly used features, however, are highly correlated. While state-of-the-art machine learning techniques can routinely filter such correlations, they hinder explainability of the derived algorithm design techniques. We therefore propose in this work to pre-process the measured (raw) landscape features through representation learning. More precisely, we show that a linear dimensionality reduction via matrix factorization significantly contributes towards a better detection of correlation between different problem instances -- a key prerequisite for successful automated algorithm design.