Linear Recurrent Units for Sequential Recommendation

TL;DR

This paper introduces LRURec, a linear recurrent unit model for sequential recommendation that achieves fast, incremental inference and reduced computational costs, outperforming current self-attention-based models.

Contribution

The paper proposes LRURec, a novel recurrent architecture optimized for efficiency and incremental inference in sequential recommendation tasks, addressing limitations of self-attention models.

Findings

LRURec outperforms state-of-the-art models on multiple datasets.

LRURec offers faster inference and training efficiency.

The architecture effectively handles long sequences with improved user experience.

Abstract

State-of-the-art sequential recommendation relies heavily on self-attention-based recommender models. Yet such models are computationally expensive and often too slow for real-time recommendation. Furthermore, the self-attention operation is performed at a sequence-level, thereby making low-cost incremental inference challenging. Inspired by recent advances in efficient language modeling, we propose linear recurrent units for sequential recommendation (LRURec). Similar to recurrent neural networks, LRURec offers rapid inference and can achieve incremental inference on sequential inputs. By decomposing the linear recurrence operation and designing recursive parallelization in our framework, LRURec provides the additional benefits of reduced model size and parallelizable training. Moreover, we optimize the architecture of LRURec by implementing a series of modifications to address the lack of non-linearity and improve training dynamics. To validate the effectiveness of our proposed LRURec, we conduct extensive experiments on multiple real-world datasets and compare its performance against state-of-the-art sequential recommenders. Experimental results demonstrate the effectiveness of LRURec, which consistently outperforms baselines by a significant margin. Results also highlight the efficiency of LRURec with our parallelized training paradigm and fast inference on long sequences, showing its potential to further enhance user experience in sequential recommendation.