LLMs as Better Recommenders with Natural Language Collaborative Signals: A Self-Assessing Retrieval Approach

TL;DR

This paper introduces SCORE, a framework that uses natural language collaborative signals and self-assessment within LLMs to improve recommendation accuracy by better aligning collaborative information with the model's semantic understanding.

Contribution

The paper proposes a novel self-assessing retrieval framework that retrieves and ranks natural language collaborative signals for LLM-based recommendation tasks.

Findings

SCORE significantly improves recommendation performance on public datasets.

Natural language collaborative signals enhance LLM understanding and relevance.

Self-assessment effectively filters and prioritizes useful collaborative behaviors.

Abstract

Incorporating collaborative information (CI) effectively is crucial for leveraging LLMs in recommendation tasks. Existing approaches often encode CI using soft tokens or abstract identifiers, which introduces a semantic misalignment with the LLM's natural language pretraining and hampers knowledge integration. To address this, we propose expressing CI directly in natural language to better align with LLMs' semantic space. We achieve this by retrieving a curated set of the most relevant user behaviors in natural language form. However, identifying informative CI is challenging due to the complexity of similarity and utility assessment. To tackle this, we introduce a Self-assessing COllaborative REtrieval framework (SCORE) following the retrieve-rerank paradigm. First, a Collaborative Retriever (CAR) is developed to consider both collaborative patterns and semantic similarity. Then, a Self-assessing Reranker (SARE) leverages LLMs' own reasoning to assess and prioritize retrieved behaviors. Finally, the selected behaviors are prepended to the LLM prompt as natural-language CI to guide recommendation. Extensive experiments on two public datasets validate the effectiveness of SCORE in improving LLM-based recommendation.