Uncovering Selective State Space Model's Capabilities in Lifelong Sequential Recommendation

TL;DR

This paper evaluates the Mamba state space model's effectiveness in lifelong sequential recommendation tasks, demonstrating its ability to maintain performance while significantly reducing computational costs.

Contribution

It introduces the use of the Mamba model for lifelong sequence recommendation and shows its advantages over existing models in efficiency and scalability.

Findings

Mamba achieves comparable recommendation accuracy to existing models.

Mamba reduces training time by approximately 70%.

Mamba decreases memory usage by about 80%.

Abstract

Sequential Recommenders have been widely applied in various online services, aiming to model users' dynamic interests from their sequential interactions. With users increasingly engaging with online platforms, vast amounts of lifelong user behavioral sequences have been generated. However, existing sequential recommender models often struggle to handle such lifelong sequences. The primary challenges stem from computational complexity and the ability to capture long-range dependencies within the sequence. Recently, a state space model featuring a selective mechanism (i.e., Mamba) has emerged. In this work, we investigate the performance of Mamba for lifelong sequential recommendation (i.e., length>=2k). More specifically, we leverage the Mamba block to model lifelong user sequences selectively. We conduct extensive experiments to evaluate the performance of representative sequential recommendation models in the setting of lifelong sequences. Experiments on two real-world datasets demonstrate the superiority of Mamba. We found that RecMamba achieves performance comparable to the representative model while significantly reducing training duration by approximately 70% and memory costs by 80%. Codes and data are available at \url{https://github.com/nancheng58/RecMamba}.