Denoising Self-attentive Sequential Recommendation

TL;DR

This paper addresses the challenge of noise in sequential recommendation sequences by proposing a denoising approach for self-attentive Transformer models, improving recommendation accuracy in noisy real-world data.

Contribution

It introduces a novel denoising mechanism for self-attention in Transformer-based recommenders, enhancing their robustness against noisy user interaction data.

Findings

Improved recommendation accuracy on noisy datasets

Enhanced robustness of Transformer models to irrelevant items

Demonstrated effectiveness over existing methods

Abstract

Transformer-based sequential recommenders are very powerful for capturing both short-term and long-term sequential item dependencies. This is mainly attributed to their unique self-attention networks to exploit pairwise item-item interactions within the sequence. However, real-world item sequences are often noisy, which is particularly true for implicit feedback. For example, a large portion of clicks do not align well with user preferences, and many products end up with negative reviews or being returned. As such, the current user action only depends on a subset of items, not on the entire sequences. Many existing Transformer-based models use full attention distributions, which inevitably assign certain credits to irrelevant items. This may lead to sub-optimal performance if Transformers are not regularized properly.