ELCoRec: Enhance Language Understanding with Co-Propagation of Numerical and Categorical Features for Recommendation

TL;DR

ELCoRec enhances language models for recommendation by co-propagating numerical, categorical, and temporal features through a GAT expert, improving understanding of user preferences and recent interests.

Contribution

The paper introduces ELCoRec, a novel method that integrates heterogeneous user features into LLMs via co-propagation and soft prompting, addressing limitations in understanding user behavior.

Findings

ELCoRec outperforms strong baselines on three datasets.

The co-propagation mechanism stabilizes heterogeneous features.

Soft prompting efficiently injects user preferences into LLMs.

Abstract

Large language models have been flourishing in the natural language processing (NLP) domain, and their potential for recommendation has been paid much attention to. Despite the intelligence shown by the recommendation-oriented finetuned models, LLMs struggle to fully understand the user behavior patterns due to their innate weakness in interpreting numerical features and the overhead for long context, where the temporal relations among user behaviors, subtle quantitative signals among different ratings, and various side features of items are not well explored. Existing works only fine-tune a sole LLM on given text data without introducing that important information to it, leaving these problems unsolved. In this paper, we propose ELCoRec to Enhance Language understanding with CoPropagation of numerical and categorical features for Recommendation. Concretely, we propose to inject the preference understanding capability into LLM via a GAT expert model where the user preference is better encoded by parallelly propagating the temporal relations, and rating signals as well as various side information of historical items. The parallel propagation mechanism could stabilize heterogeneous features and offer an informative user preference encoding, which is then injected into the language models via soft prompting at the cost of a single token embedding. To further obtain the user's recent interests, we proposed a novel Recent interaction Augmented Prompt (RAP) template. Experiment results over three datasets against strong baselines validate the effectiveness of ELCoRec. The code is available at https://anonymous.4open.science/r/CIKM_Code_Repo-E6F5/README.md.