Predictive accuracy of recommender algorithms

TL;DR

This paper compares the predictive accuracy of traditional and deep learning recommender algorithms using controlled experiments on benchmark datasets, highlighting challenges and potential improvements for deep learning approaches.

Contribution

It provides a systematic comparison of conventional and deep learning recommender algorithms under controlled conditions, revealing current limitations of deep learning methods.

Findings

Non-deep learning algorithms performed as expected based on prior benchmarks.

Deep learning algorithms underperformed, likely due to overfitting issues.

Regularization strategies may enhance deep learning recommender accuracy.

Abstract

Recommender systems present a customized list of items based upon user or item characteristics with the objective of reducing a large number of possible choices to a smaller ranked set most likely to appeal to the user. A variety of algorithms for recommender systems have been developed and refined including applications of deep learning neural networks. Recent research reports point to a need to perform carefully controlled experiments to gain insights about the relative accuracy of different recommender algorithms, because studies evaluating different methods have not used a common set of benchmark data sets, baseline models, and evaluation metrics. This investigation used publicly available sources of ratings data with a suite of three conventional recommender algorithms and two deep learning (DL) algorithms in controlled experiments to assess their comparative accuracy. Results for the non-DL algorithms conformed well to published results and benchmarks. The two DL algorithms did not perform as well and illuminated known challenges implementing DL recommender algorithms as reported in the literature. Model overfitting is discussed as a potential explanation for the weaker performance of the DL algorithms and several regularization strategies are reviewed as possible approaches to improve predictive error. Findings justify the need for further research in the use of deep learning models for recommender systems.