CoLLM: Integrating Collaborative Embeddings into Large Language Models for Recommendation

TL;DR

CoLLM introduces a novel method to incorporate collaborative user-item interaction data into large language models for improved recommendations across cold and warm start scenarios, without altering the core LLM architecture.

Contribution

It presents a new approach that externalizes collaborative information into LLMs, enhancing recommendation quality without modifying the models themselves.

Findings

Improved recommendation accuracy in cold and warm start scenarios.

Effective integration of collaborative data without changing LLM architecture.

Flexible framework for incorporating various collaborative models.

Abstract

Leveraging Large Language Models as Recommenders (LLMRec) has gained significant attention and introduced fresh perspectives in user preference modeling. Existing LLMRec approaches prioritize text semantics, usually neglecting the valuable collaborative information from user-item interactions in recommendations. While these text-emphasizing approaches excel in cold-start scenarios, they may yield sub-optimal performance in warm-start situations. In pursuit of superior recommendations for both cold and warm start scenarios, we introduce CoLLM, an innovative LLMRec methodology that seamlessly incorporates collaborative information into LLMs for recommendation. CoLLM captures collaborative information through an external traditional model and maps it to the input token embedding space of LLM, forming collaborative embeddings for LLM usage. Through this external integration of collaborative information, CoLLM ensures effective modeling of collaborative information without modifying the LLM itself, providing the flexibility to employ various collaborative information modeling techniques. Extensive experiments validate that CoLLM adeptly integrates collaborative information into LLMs, resulting in enhanced recommendation performance. We release the code and data at https://github.com/zyang1580/CoLLM.