Mutual Wasserstein Discrepancy Minimization for Sequential Recommendation

TL;DR

This paper introduces MStein, a novel self-supervised sequential recommendation framework that uses Wasserstein Discrepancy Measurement to improve mutual information estimation, robustness, and training efficiency over traditional KL-based methods.

Contribution

It proposes a new mutual information measurement based on Wasserstein distance, addressing limitations of KL divergence and enhancing robustness and efficiency in sequential recommendation.

Findings

MStein outperforms baseline methods on benchmark datasets.

Demonstrates robustness against data perturbations.

Shows improved training efficiency with smaller batch sizes.

Abstract

Self-supervised sequential recommendation significantly improves recommendation performance by maximizing mutual information with well-designed data augmentations. However, the mutual information estimation is based on the calculation of Kullback Leibler divergence with several limitations, including asymmetrical estimation, the exponential need of the sample size, and training instability. Also, existing data augmentations are mostly stochastic and can potentially break sequential correlations with random modifications. These two issues motivate us to investigate an alternative robust mutual information measurement capable of modeling uncertainty and alleviating KL divergence limitations. To this end, we propose a novel self-supervised learning framework based on Mutual WasserStein discrepancy minimization MStein for the sequential recommendation. We propose the Wasserstein Discrepancy Measurement to measure the mutual information between augmented sequences. Wasserstein Discrepancy Measurement builds upon the 2-Wasserstein distance, which is more robust, more efficient in small batch sizes, and able to model the uncertainty of stochastic augmentation processes. We also propose a novel contrastive learning loss based on Wasserstein Discrepancy Measurement. Extensive experiments on four benchmark datasets demonstrate the effectiveness of MStein over baselines. More quantitative analyses show the robustness against perturbations and training efficiency in batch size. Finally, improvements analysis indicates better representations of popular users or items with significant uncertainty. The source code is at https://github.com/zfan20/MStein.