Think before Recommendation: Autonomous Reasoning-enhanced Recommender

TL;DR

This paper introduces RecZero, an RL-based approach that trains a single LLM to develop autonomous reasoning capabilities for recommendation tasks, outperforming existing methods by avoiding distillation limitations.

Contribution

Proposes RecZero, a novel RL-based framework for training LLMs to autonomously reason for recommendations, eliminating the need for multi-model distillation.

Findings

RecZero outperforms baseline methods on benchmark datasets.

RecOne, a hybrid model, further improves performance with supervised fine-tuning.

Autonomous reasoning enhances recommendation accuracy.

Abstract

The core task of recommender systems is to learn user preferences from historical user-item interactions. With the rapid development of large language models (LLMs), recent research has explored leveraging the reasoning capabilities of LLMs to enhance rating prediction tasks. However, existing distillation-based methods suffer from limitations such as the teacher model's insufficient recommendation capability, costly and static supervision, and superficial transfer of reasoning ability. To address these issues, this paper proposes RecZero, a reinforcement learning (RL)-based recommendation paradigm that abandons the traditional multi-model and multi-stage distillation approach. Instead, RecZero trains a single LLM through pure RL to autonomously develop reasoning capabilities for rating prediction. RecZero consists of two key components: (1) "Think-before-Recommendation" prompt construction, which employs a structured reasoning template to guide the model in step-wise analysis of user interests, item features, and user-item compatibility; and (2) rule-based reward modeling, which adopts group relative policy optimization (GRPO) to compute rewards for reasoning trajectories and optimize the LLM. Additionally, the paper explores a hybrid paradigm, RecOne, which combines supervised fine-tuning with RL, initializing the model with cold-start reasoning samples and further optimizing it with RL. Experimental results demonstrate that RecZero and RecOne significantly outperform existing baseline methods on multiple benchmark datasets, validating the superiority of the RL paradigm in achieving autonomous reasoning-enhanced recommender systems.