Algorithm Selection for Collaborative Filtering: the influence of graph metafeatures and multicriteria metatargets

TL;DR

This paper introduces new graph-based metafeatures and investigates multi-dimensional metafeatures and multicriteria metatargets to improve algorithm selection in collaborative filtering, with experimental validation showing promising results for graph features.

Contribution

It proposes novel graph metafeatures and a multicriteria ranking method, enhancing the understanding of feature effectiveness in collaborative filtering algorithm selection.

Findings

Graph metafeatures are a viable alternative to existing features.

Feature selection for comprehensive metafeatures showed no performance gain.

Multicriteria metatargets enable more nuanced algorithm ranking.

Abstract

To select the best algorithm for a new problem is an expensive and difficult task. However, there are automatic solutions to address this problem: using Metalearning, which takes advantage of problem characteristics (i.e. metafeatures), one is able to predict the relative performance of algorithms. In the Collaborative Filtering scope, recent works have proposed diverse metafeatures describing several dimensions of this problem. Despite interesting and effective findings, it is still unknown whether these are the most effective metafeatures. Hence, this work proposes a new set of graph metafeatures, which approach the Collaborative Filtering problem from a Graph Theory perspective. Furthermore, in order to understand whether metafeatures from multiple dimensions are a better fit, we investigate the effects of comprehensive metafeatures. These metafeatures are a selection of the best metafeatures from all existing Collaborative Filtering metafeatures. The impact of the most representative metafeatures is investigated in a controlled experimental setup. Another contribution we present is the use of a Pareto-Efficient ranking procedure to create multicriteria metatargets. These new rankings of algorithms, which take into account multiple evaluation measures, allow to explore the algorithm selection problem in a fairer and more detailed way. According to the experimental results, the graph metafeatures are a good alternative to related work metafeatures. However, the results have shown that the feature selection procedure used to create the comprehensive metafeatures is is not effective, since there is no gain in predictive performance. Finally, an extensive metaknowledge analysis was conducted to identify the most influential metafeatures.