A Systematic Replicability and Comparative Study of BSARec and SASRec for Sequential Recommendation

TL;DR

This paper systematically compares BSARec and SASRec for sequential recommendation, highlighting how BSARec's frequency enhancement improves performance, but emphasizing the importance of implementation details for fair evaluation.

Contribution

It provides a fair, reproducible comparison of BSARec and SASRec, demonstrating the impact of frequency enhancement and emphasizing implementation details in performance evaluation.

Findings

BSARec outperforms SASRec with frequency enhancement.

Performance gains are modest compared to original claims.

Implementation details significantly affect comparative results.

Abstract

This study aims at comparing two sequential recommender systems: Self-Attention based Sequential Recommendation (SASRec), and Beyond Self-Attention based Sequential Recommendation (BSARec) in order to check the improvement frequency enhancement - the added element in BSARec - has on recommendations. The models in the study, have been re-implemented with a common base-structure from EasyRec, with the aim of obtaining a fair and reproducible comparison. The results obtained displayed how BSARec, by including bias terms for frequency enhancement, does indeed outperform SASRec, although the increases in performance obtained, are not as high as those presented by the authors. This work aims at offering an overview on existing methods, and most importantly at underlying the importance of implementation details for performance comparison.