STAR-Rec: Making Peace with Length Variance and Pattern Diversity in Sequential Recommendation

TL;DR

STAR-Rec introduces a novel sequential recommendation architecture that effectively models user behavior diversity and sequence length variations by combining preference-aware attention, state-space modeling, and a mixture-of-experts framework, leading to superior performance.

Contribution

The paper presents STAR-Rec, a new model integrating attention, state-space models, and mixture-of-experts to better handle diversity and length variability in sequential recommendation.

Findings

STAR-Rec outperforms existing methods on four real-world datasets.

It effectively captures diverse user interaction patterns.

The model handles variable sequence lengths with linear complexity.

Abstract

Recent deep sequential recommendation models often struggle to effectively model key characteristics of user behaviors, particularly in handling sequence length variations and capturing diverse interaction patterns. We propose STAR-Rec, a novel architecture that synergistically combines preference-aware attention and state-space modeling through a sequence-level mixture-of-experts framework. STAR-Rec addresses these challenges by: (1) employing preference-aware attention to capture both inherently similar item relationships and diverse preferences, (2) utilizing state-space modeling to efficiently process variable-length sequences with linear complexity, and (3) incorporating a mixture-of-experts component that adaptively routes different behavioral patterns to specialized experts, handling both focused category-specific browsing and diverse category exploration patterns. We theoretically demonstrate how the state space model and attention mechanisms can be naturally unified in recommendation scenarios, where SSM captures temporal dynamics through state compression while attention models both similar and diverse item relationships. Extensive experiments on four real-world datasets demonstrate that STAR-Rec consistently outperforms state-of-the-art sequential recommendation methods, particularly in scenarios involving diverse user behaviors and varying sequence lengths.