Session-aware Recommendation: A Surprising Quest for the State-of-the-art

TL;DR

This paper benchmarks session-aware recommendation algorithms and finds that simple nearest neighbor methods often outperform complex neural models, highlighting the need for better evaluation practices and more sophisticated methods.

Contribution

It provides a comprehensive benchmarking of session-aware algorithms against session-based and trivial methods, revealing surprising performance insights.

Findings

Nearest neighbor methods outperform neural techniques

Session-aware models are not significantly better than session-based approaches

Many existing comparisons use weak baselines, indicating methodological issues

Abstract

Recommender systems are designed to help users in situations of information overload. In recent years, we observed increased interest in session-based recommendation scenarios, where the problem is to make item suggestions to users based only on interactions observed in an ongoing session. However, in cases where interactions from previous user sessions are available, the recommendations can be personalized according to the users' long-term preferences, a process called session-aware recommendation. Today, research in this area is scattered and many existing works only compare session-aware with session-based models. This makes it challenging to understand what represents the state-of-the-art. To close this research gap, we benchmarked recent session-aware algorithms against each other and against a number of session-based recommendation algorithms and trivial extensions thereof. Our comparison, to some surprise, revealed that (i) item simple techniques based on nearest neighbors consistently outperform recent neural techniques and that (ii) session-aware models were mostly not better than approaches that do not use long-term preference information. Our work therefore not only points to potential methodological issues where new methods are compared to weak baselines, but also indicates that there remains a huge potential for more sophisticated session-aware recommendation algorithms.