Recurrent Preference Memory for Efficient Long-Sequence Generative Recommendation

TL;DR

Rec2PM introduces a memory compression framework for generative recommendation that enables efficient, parallel training and inference on long user sequences, reducing latency and memory use while improving accuracy.

Contribution

It proposes a novel Preference Memory framework with a self-referential teacher-forcing strategy for scalable, parallel training of long-sequence recommendation models.

Findings

Significantly reduces inference latency and memory footprint.

Achieves superior accuracy over full-sequence models.

Effectively filters noise to capture robust long-term interests.

Abstract

Generative recommendation (GenRec) models typically model user behavior via full attention, but scaling to lifelong sequences is hindered by prohibitive computational costs and noise accumulation from stochastic interactions. To address these challenges, we introduce Rec2PM, a framework that compresses long user interaction histories into compact Preference Memory tokens. Unlike traditional recurrent methods that suffer from serial training, Rec2PM employs a novel self-referential teacher-forcing strategy: it leverages a global view of the history to generate reference memories, which serve as supervision targets for parallelized recurrent updates. This allows for fully parallel training while maintaining the capability for iterative updates during inference. Additionally, by representing memory as token embeddings rather than extensive KV caches, Rec2PM achieves extreme storage efficiency. Experiments on large-scale benchmarks show that Rec2PM significantly reduces inference latency and memory footprint while achieving superior accuracy compared to full-sequence models. Analysis reveals that the Preference Memory functions as a denoising Information Bottleneck, effectively filtering interaction noise to capture robust long-term interests.