A Practice-Friendly LLM-Enhanced Paradigm with Preference Parsing for Sequential Recommendation

TL;DR

This paper introduces P2Rec, a new LLM-enhanced paradigm for sequential recommendation that improves efficiency and effectiveness by using preference parsing and user-level fine-tuning, reducing reliance on textual data.

Contribution

Proposes a practice-friendly LLM-enhanced framework with preference parsing for SRS, utilizing user-level fine-tuning and information augmentation to improve efficiency and effectiveness.

Findings

P2Rec outperforms existing methods on benchmark datasets.

The new user-level SFT task effectively injects collaborative information.

Enhanced embeddings improve subsequent SRS model training.

Abstract

The training paradigm integrating large language models (LLM) is gradually reshaping sequential recommender systems (SRS) and has shown promising results. However, most existing LLM-enhanced methods rely on rich textual information on the item side and instance-level supervised fine-tuning (SFT) to inject collaborative information into LLM, which is inefficient and limited in many applications. To alleviate these problems, this paper proposes a practice-friendly LLM-enhanced paradigm with preference parsing (P2Rec) for SRS. Specifically, in the information reconstruction stage, we design a new user-level SFT task for collaborative information injection with the assistance of a pre-trained SRS model, which is more efficient and compatible with limited text information. Our goal is to let LLM learn to reconstruct a corresponding prior preference distribution from each user's interaction sequence, where LLM needs to effectively parse the latent category of each item and the relationship between different items to accomplish this task. In the information augmentation stage, we feed each item into LLM to obtain a set of enhanced embeddings that combine collaborative information and LLM inference capabilities. These embeddings can then be used to help train various future SRS models. Finally, we verify the effectiveness and efficiency of our TSLRec on three SRS benchmark datasets.