Modeling Sequences as Distributions with Uncertainty for Sequential Recommendation

TL;DR

This paper introduces DT4SR, a distribution-based Transformer model that incorporates uncertainty into sequential recommendation, effectively capturing stochastic user behaviors and alleviating cold-start problems.

Contribution

It proposes a novel distribution-based Transformer using Elliptical Gaussian distributions and Wasserstein distance for uncertainty modeling in sequential recommendation.

Findings

Outperforms state-of-the-art methods on benchmark datasets

Effectively alleviates cold-start issues

Demonstrates the importance of modeling uncertainty in sequences

Abstract

The sequential patterns within the user interactions are pivotal for representing the user's preference and capturing latent relationships among items. The recent advancements of sequence modeling by Transformers advocate the community to devise more effective encoders for the sequential recommendation. Most existing sequential methods assume users are deterministic. However, item-item transitions might fluctuate significantly in several item aspects and exhibit randomness of user interests. This \textit{stochastic characteristics} brings up a solid demand to include uncertainties in representing sequences and items. Additionally, modeling sequences and items with uncertainties expands users' and items' interaction spaces, thus further alleviating cold-start problems. In this work, we propose a Distribution-based Transformer for Sequential Recommendation (DT4SR), which injects uncertainties into sequential modeling. We use Elliptical Gaussian distributions to describe items and sequences with uncertainty. We describe the uncertainty in items and sequences as Elliptical Gaussian distribution. And we adopt Wasserstein distance to measure the similarity between distributions. We devise two novel Trans-formers for modeling mean and covariance, which guarantees the positive-definite property of distributions. The proposed method significantly outperforms the state-of-the-art methods. The experiments on three benchmark datasets also demonstrate its effectiveness in alleviating cold-start issues. The code is available inhttps://github.com/DyGRec/DT4SR.