Collaborative Word-based Pre-trained Item Representation for Transferable Recommendation

TL;DR

This paper introduces CoWPiRec, a novel method that enhances item representations for recommendation systems by integrating collaborative filtering information into pre-trained language models, improving transferability and cold-start performance.

Contribution

It proposes a new approach to incorporate collaborative filtering data into text-based item representations using a word graph and a specialized pre-training task.

Findings

Outperforms state-of-the-art transfer recommenders in multiple datasets.

Improves cold-start recommendation effectiveness.

Enhances zero-shot cross-scenario recommendation performance.

Abstract

Item representation learning (IRL) plays an essential role in recommender systems, especially for sequential recommendation. Traditional sequential recommendation models usually utilize ID embeddings to represent items, which are not shared across different domains and lack the transferable ability. Recent studies use pre-trained language models (PLM) for item text embeddings (text-based IRL) that are universally applicable across domains. However, the existing text-based IRL is unaware of the important collaborative filtering (CF) information. In this paper, we propose CoWPiRec, an approach of Collaborative Word-based Pre-trained item representation for Recommendation. To effectively incorporate CF information into text-based IRL, we convert the item-level interaction data to a word graph containing word-level collaborations. Subsequently, we design a novel pre-training task to align the word-level semantic- and CF-related item representation. Extensive experimental results on multiple public datasets demonstrate that compared to state-of-the-art transferable sequential recommenders, CoWPiRec achieves significantly better performances in both fine-tuning and zero-shot settings for cross-scenario recommendation and effectively alleviates the cold-start issue. The code is available at: https://github.com/ysh-1998/CoWPiRec.