gSASRec: Reducing Overconfidence in Sequential Recommendation Trained with Negative Sampling

TL;DR

This paper introduces gSASRec, a recommendation model that reduces overconfidence caused by negative sampling using a new loss function, leading to improved performance and efficiency on large datasets.

Contribution

The paper proposes a novel gBCE loss to mitigate overconfidence in sequential recommendation models trained with negative sampling, and develops gSASRec, an improved model that outperforms existing methods.

Findings

gSASRec outperforms BERT4Rec in NDCG by 9.47% on MovieLens-1M.

gSASRec reduces training time by 73% on MovieLens-1M.

gSASRec is scalable to datasets with over 1 million items.

Abstract

A large catalogue size is one of the central challenges in training recommendation models: a large number of items makes them memory and computationally inefficient to compute scores for all items during training, forcing these models to deploy negative sampling. However, negative sampling increases the proportion of positive interactions in the training data, and therefore models trained with negative sampling tend to overestimate the probabilities of positive interactions a phenomenon we call overconfidence. While the absolute values of the predicted scores or probabilities are not important for the ranking of retrieved recommendations, overconfident models may fail to estimate nuanced differences in the top-ranked items, resulting in degraded performance. In this paper, we show that overconfidence explains why the popular SASRec model underperforms when compared to BERT4Rec. This is contrary to the BERT4Rec authors explanation that the difference in performance is due to the bi-directional attention mechanism. To mitigate overconfidence, we propose a novel Generalised Binary Cross-Entropy Loss function (gBCE) and theoretically prove that it can mitigate overconfidence. We further propose the gSASRec model, an improvement over SASRec that deploys an increased number of negatives and the gBCE loss. We show through detailed experiments on three datasets that gSASRec does not exhibit the overconfidence problem. As a result, gSASRec can outperform BERT4Rec (e.g. +9.47% NDCG on the MovieLens-1M dataset), while requiring less training time (e.g. -73% training time on MovieLens-1M). Moreover, in contrast to BERT4Rec, gSASRec is suitable for large datasets that contain more than 1 million items.