Comprehensive Evaluation of Matrix Factorization Models for Collaborative Filtering Recommender Systems

TL;DR

This paper comprehensively evaluates six matrix factorization models across multiple datasets and metrics, providing insights into their performance, interpretability, and suitability for various recommendation scenarios.

Contribution

It offers a systematic comparison of matrix factorization models using diverse quality measures and promotes reproducibility with an open framework and code.

Findings

Different models excel in various accuracy and diversity metrics.

Simplicity of models influences interpretability and computational efficiency.

Recommendations for model selection depend on specific application needs.

Abstract

Matrix factorization models are the core of current commercial collaborative filtering Recommender Systems. This paper tested six representative matrix factorization models, using four collaborative filtering datasets. Experiments have tested a variety of accuracy and beyond accuracy quality measures, including prediction, recommendation of ordered and unordered lists, novelty, and diversity. Results show each convenient matrix factorization model attending to their simplicity, the required prediction quality, the necessary recommendation quality, the desired recommendation novelty and diversity, the need to explain recommendations, the adequacy of assigning semantic interpretations to hidden factors, the advisability of recommending to groups of users, and the need to obtain reliability values. To ensure the reproducibility of the experiments, an open framework has been used, and the implementation code is provided.