Evaluating Meta-Feature Selection for the Algorithm Recommendation Problem

TL;DR

This paper empirically evaluates feature selection and extraction techniques for meta-features in algorithm recommendation, finding that dimensionality reduction can reduce features significantly without sacrificing predictive performance, thus improving efficiency.

Contribution

It provides an empirical analysis of feature selection and extraction methods for meta-features in algorithm recommendation, highlighting effective strategies for dimensionality reduction.

Findings

Dimensionality reduction reduces about 80% of meta-features.

High predictive performance can be achieved with around 20% of original meta-features.

DR methods generally do not improve predictive performance but lower runtime.

Abstract

With the popularity of Machine Learning (ML) solutions, algorithms and data have been released faster than the capacity of processing them. In this context, the problem of Algorithm Recommendation (AR) is receiving a significant deal of attention recently. This problem has been addressed in the literature as a learning task, often as a Meta-Learning problem where the aim is to recommend the best alternative for a specific dataset. For such, datasets encoded by meta-features are explored by ML algorithms that try to learn the mapping between meta-representations and the best technique to be used. One of the challenges for the successful use of ML is to define which features are the most valuable for a specific dataset since several meta-features can be used, which increases the meta-feature dimension. This paper presents an empirical analysis of Feature Selection and Feature Extraction in the meta-level for the AR problem. The present study was focused on three criteria: predictive performance, dimensionality reduction, and pipeline runtime. As we verified, applying Dimensionality Reduction (DR) methods did not improve predictive performances in general. However, DR solutions reduced about 80% of the meta-features, obtaining pretty much the same performance as the original setup but with lower runtimes. The only exception was PCA, which presented about the same runtime as the original meta-features. Experimental results also showed that various datasets have many non-informative meta-features and that it is possible to obtain high predictive performance using around 20% of the original meta-features. Therefore, due to their natural trend for high dimensionality, DR methods should be used for Meta-Feature Selection and Meta-Feature Extraction.