Simplify and Robustify Negative Sampling for Implicit Collaborative Filtering

TL;DR

This paper introduces a simplified and robust negative sampling method for implicit collaborative filtering, focusing on selecting high-variance, important negative samples to improve efficiency and effectiveness in recommendation models.

Contribution

The paper proposes a novel negative sampling approach that simplifies the process by using a memory of important candidates and addresses false negatives by favoring high-variance samples.

Findings

Improved robustness and accuracy over existing methods

Effective sampling of true negatives with high quality

Validated on multiple synthetic and real-world datasets

Abstract

Negative sampling approaches are prevalent in implicit collaborative filtering for obtaining negative labels from massive unlabeled data. As two major concerns in negative sampling, efficiency and effectiveness are still not fully achieved by recent works that use complicate structures and overlook risk of false negative instances. In this paper, we first provide a novel understanding of negative instances by empirically observing that only a few instances are potentially important for model learning, and false negatives tend to have stable predictions over many training iterations. Above findings motivate us to simplify the model by sampling from designed memory that only stores a few important candidates and, more importantly, tackle the untouched false negative problem by favouring high-variance samples stored in memory, which achieves efficient sampling of true negatives with high-quality. Empirical results on two synthetic datasets and three real-world datasets demonstrate both robustness and superiorities of our negative sampling method.