ID-Agnostic User Behavior Pre-training for Sequential Recommendation

TL;DR

This paper introduces IDA-SR, a novel ID-agnostic pre-training method for sequential recommendation that leverages rich text information and pre-trained language models to improve recommendation performance without relying solely on item IDs.

Contribution

The paper proposes a new ID-agnostic pre-training approach using text-based item representations and pre-trained language models for sequential recommendation tasks.

Findings

Achieves comparable results with ID-based methods using only text representations.

Outperforms baseline models significantly after fine-tuning with ID information.

Demonstrates effectiveness of text-based pre-training in recommendation systems.

Abstract

Recently, sequential recommendation has emerged as a widely studied topic. Existing researches mainly design effective neural architectures to model user behavior sequences based on item IDs. However, this kind of approach highly relies on user-item interaction data and neglects the attribute- or characteristic-level correlations among similar items preferred by a user. In light of these issues, we propose IDA-SR, which stands for ID-Agnostic User Behavior Pre-training approach for Sequential Recommendation. Instead of explicitly learning representations for item IDs, IDA-SR directly learns item representations from rich text information. To bridge the gap between text semantics and sequential user behaviors, we utilize the pre-trained language model as text encoder, and conduct a pre-training architecture on the sequential user behaviors. In this way, item text can be directly utilized for sequential recommendation without relying on item IDs. Extensive experiments show that the proposed approach can achieve comparable results when only using ID-agnostic item representations, and performs better than baselines by a large margin when fine-tuned with ID information.