R4ec: A Reasoning, Reflection, and Refinement Framework for Recommendation Systems

TL;DR

This paper introduces R4ec, a framework that enhances recommendation systems by enabling large language models to perform iterative reasoning, reflection, and refinement, leading to more accurate predictions and increased revenue.

Contribution

The paper proposes R4ec, a novel System-2 inspired framework with actor and reflection models for improved recommendation accuracy and robustness.

Findings

R4ec outperforms baseline models on Amazon-Book and MovieLens-1M datasets.

Deployment of R4ec on an online platform increased revenue by 2.2%.

The framework demonstrates scalable benefits with larger models.

Abstract

Harnessing Large Language Models (LLMs) for recommendation systems has emerged as a prominent avenue, drawing substantial research interest. However, existing approaches primarily involve basic prompt techniques for knowledge acquisition, which resemble System-1 thinking. This makes these methods highly sensitive to errors in the reasoning path, where even a small mistake can lead to an incorrect inference. To this end, in this paper, we propose $R^{4}$ec, a reasoning, reflection and refinement framework that evolves the recommendation system into a weak System-2 model. Specifically, we introduce two models: an actor model that engages in reasoning, and a reflection model that judges these responses and provides valuable feedback. Then the actor model will refine its response based on the feedback, ultimately leading to improved responses. We employ an iterative reflection and refinement process, enabling LLMs to facilitate slow and deliberate System-2-like thinking. Ultimately, the final refined knowledge will be incorporated into a recommendation backbone for prediction. We conduct extensive experiments on Amazon-Book and MovieLens-1M datasets to demonstrate the superiority of $R^{4}$ec. We also deploy $R^{4}$ec on a large scale online advertising platform, showing 2.2\% increase of revenue. Furthermore, we investigate the scaling properties of the actor model and reflection model.