Reasoning Meets Personalization: Unleashing the Potential of Large Reasoning Model for Personalized Generation

TL;DR

This paper evaluates large reasoning models for personalization, identifies key limitations, and proposes a novel framework with hierarchical reasoning and intervention methods to improve personalized generation performance.

Contribution

It introduces Reinforced Reasoning for Personalization ( extit{ReaPer}), a framework that enhances LRM personalization through structured reasoning templates and alignment techniques.

Findings

LRMs do not always outperform general LLMs in retrieval scenarios

The proposed extit{ReaPer} framework significantly improves personalization quality

Hierarchical reasoning and intervention methods enhance model alignment and consistency

Abstract

Personalization is a critical task in modern intelligent systems, with applications spanning diverse domains, including interactions with large language models (LLMs). Recent advances in reasoning capabilities have significantly enhanced LLMs, enabling unprecedented performance in tasks such as mathematics and coding. However, their potential for personalization tasks remains underexplored. In this paper, we present the first systematic evaluation of large reasoning models (LRMs) for personalization tasks. Surprisingly, despite generating more tokens, LRMs do not consistently outperform general-purpose LLMs, especially in retrieval-intensive scenarios where their advantages diminish. Our analysis identifies three key limitations: divergent thinking, misalignment of response formats, and ineffective use of retrieved information. To address these challenges, we propose Reinforced Reasoning for Personalization (\model), a novel framework that incorporates a hierarchical reasoning thought template to guide LRMs in generating structured outputs. Additionally, we introduce a reasoning process intervention method to enforce adherence to designed reasoning patterns, enhancing alignment. We also propose a cross-referencing mechanism to ensure consistency. Extensive experiments demonstrate that our approach significantly outperforms existing techniques.