Evaluation of Session-based Recommendation Algorithms

TL;DR

This paper compares various session-based recommendation algorithms, revealing that simpler methods often outperform complex neural network models, highlighting the need for further innovation in this area.

Contribution

It provides an extensive performance comparison of session-based recommenders, including recent neural and simpler approaches, across multiple datasets and evaluation metrics.

Findings

Simpler methods often match or outperform neural network approaches.

There is significant potential for developing more advanced session-based algorithms.

Neural models do not always provide superior performance in session-based recommendations.

Abstract

Recommender systems help users find relevant items of interest, for example on e-commerce or media streaming sites. Most academic research is concerned with approaches that personalize the recommendations according to long-term user profiles. In many real-world applications, however, such long-term profiles often do not exist and recommendations therefore have to be made solely based on the observed behavior of a user during an ongoing session. Given the high practical relevance of the problem, an increased interest in this problem can be observed in recent years, leading to a number of proposals for session-based recommendation algorithms that typically aim to predict the user's immediate next actions. In this work, we present the results of an in-depth performance comparison of a number of such algorithms, using a variety of datasets and evaluation measures. Our comparison includes the most recent approaches based on recurrent neural networks like GRU4REC, factorized Markov model approaches such as FISM or FOSSIL, as well as simpler methods based, e.g., on nearest neighbor schemes. Our experiments reveal that algorithms of this latter class, despite their sometimes almost trivial nature, often perform equally well or significantly better than today's more complex approaches based on deep neural networks. Our results therefore suggest that there is substantial room for improvement regarding the development of more sophisticated session-based recommendation algorithms.