Towards Comprehensible Recommendation with Large Language Model Fine-tuning

TL;DR

This paper introduces CURec, a framework that fine-tunes large language models with reinforcement learning to generate accurate, personalized, and comprehensible recommendation reasons, bridging the semantic gap in recommender systems.

Contribution

It proposes a novel fine-tuning approach for LLMs using reward models and RL to improve recommendation explanations and performance.

Findings

CURec outperforms existing methods on public benchmarks.

Fine-tuning with reward signals enhances explanation accuracy.

The approach improves recommendation comprehensibility and effectiveness.

Abstract

Recommender systems have become increasingly ubiquitous in daily life. While traditional recommendation approaches primarily rely on ID-based representations or item-side content features, they often fall short in capturing the underlying semantics aligned with user preferences (e.g., recommendation reasons for items), leading to a semantic-collaborative gap. Recently emerged LLM-based feature extraction approaches also face a key challenge: how to ensure that LLMs possess recommendation-aligned reasoning capabilities and can generate accurate, personalized reasons to mitigate the semantic-collaborative gap. To address these issues, we propose a novel Content Understanding from a Collaborative Perspective framework (CURec), which generates collaborative-aligned content features for more comprehensive recommendations. \method first aligns the LLM with recommendation objectives through pretraining, equipping it with instruction-following and chain-of-thought reasoning capabilities. Next, we design a reward model inspired by traditional recommendation architectures to evaluate the quality of the recommendation reasons generated by the LLM. Finally, using the reward signals, CURec fine-tunes the LLM through RL and corrects the generated reasons to ensure their accuracy. The corrected reasons are then integrated into a downstream recommender model to enhance comprehensibility and recommendation performance. Extensive experiments on public benchmarks demonstrate the superiority of CURec over existing methods.