Text-like Encoding of Collaborative Information in Large Language Models for Recommendation

TL;DR

This paper introduces BinLLM, a novel method for integrating collaborative information into large language models for recommendation by encoding it in a text-like binary format, improving alignment and performance.

Contribution

BinLLM presents a new text-like binary encoding approach for collaborative information, enabling better integration with LLMs in recommendation systems.

Findings

BinLLM improves recommendation performance over existing methods.

Binary encoding aligns collaborative info more effectively with LLMs.

Compression options reduce sequence length without sacrificing accuracy.

Abstract

When adapting Large Language Models for Recommendation (LLMRec), it is crucial to integrate collaborative information. Existing methods achieve this by learning collaborative embeddings in LLMs' latent space from scratch or by mapping from external models. However, they fail to represent the information in a text-like format, which may not align optimally with LLMs. To bridge this gap, we introduce BinLLM, a novel LLMRec method that seamlessly integrates collaborative information through text-like encoding. BinLLM converts collaborative embeddings from external models into binary sequences -- a specific text format that LLMs can understand and operate on directly, facilitating the direct usage of collaborative information in text-like format by LLMs. Additionally, BinLLM provides options to compress the binary sequence using dot-decimal notation to avoid excessively long lengths. Extensive experiments validate that BinLLM introduces collaborative information in a manner better aligned with LLMs, resulting in enhanced performance. We release our code at https://github.com/zyang1580/BinLLM.