Capturing User Interests from Data Streams for Continual Sequential Recommendation

TL;DR

This paper introduces CSTRec, a continual learning-based transformer model for sequential recommendation that effectively captures evolving user interests over time while mitigating forgetting and maintaining high recommendation accuracy.

Contribution

We propose CSTRec with Continual Sequential Attention, including Cauchy-Schwarz Normalization and Interest Enrichment, to improve long-term interest modeling in continual sequential recommendation.

Findings

CSTRec outperforms existing models in real-world datasets.

It effectively balances knowledge retention and new interest acquisition.

The model demonstrates robustness to user interest shifts.

Abstract

Transformer-based sequential recommendation (SR) models excel at modeling long-range dependencies in user behavior via self-attention. However, updating them with continuously arriving behavior sequences incurs high computational costs or leads to catastrophic forgetting. Although continual learning, a standard approach for non-stationary data streams, has recently been applied to recommendation, existing methods gradually forget long-term user preferences and remain underexplored in SR. In this paper, we introduce Continual Sequential Transformer for Recommendation (CSTRec). CSTRec is designed to effectively adapt to current interests by leveraging well-preserved historical ones, thus capturing the trajectory of user interests over time. The core of CSTRec is Continual Sequential Attention (CSA), a linear attention tailored for continual SR, which enables CSTRec to partially retain historical knowledge without direct access to prior data. CSA has two key components: (1) Cauchy-Schwarz Normalization that stabilizes learning over time under uneven user interaction frequencies; (2) Collaborative Interest Enrichment that alleviates forgetting through shared, learnable interest pools. In addition, we introduce a new technique to facilitate the adaptation of new users by transferring historical knowledge from existing users with similar interests. Extensive experiments on three real-world datasets show that CSTRec outperforms state-of-the-art models in both knowledge retention and acquisition.