AgenticRec: End-to-End Tool-Integrated Policy Optimization for Ranking-Oriented Recommender Agents

TL;DR

AgenticRec is a comprehensive framework that enhances recommender agents by integrating reasoning, tool use, and ranking optimization, leading to improved recommendation quality through novel policy optimization and preference refinement techniques.

Contribution

It introduces a unified agentic recommendation framework with a new tool integration, unbiased ranking optimization, and fine-grained preference refinement, advancing the state of recommender agents.

Findings

Significantly outperforms baseline methods on benchmark datasets.

Effectively unifies reasoning, tool use, and ranking optimization.

Improves recommendation accuracy and preference modeling.

Abstract

Recommender agents built on Large Language Models offer a promising paradigm for recommendation. However, existing recommender agents typically suffer from a disconnect between intermediate reasoning and final ranking feedback, and are unable to capture fine-grained preferences. To address this, we present AgenticRec, a ranking-oriented agentic recommendation framework that optimizes the entire decision-making trajectory (including intermediate reasoning, tool invocation, and final ranking list generation) under sparse implicit feedback. Our approach makes three key contributions. First, we design a suite of recommendation-specific tools integrated into a ReAct loop to support evidence-grounded reasoning. Second, we propose theoretically unbiased List-Wise Group Relative Policy Optimization (list-wise GRPO) to maximize ranking utility, ensuring accurate credit assignment for complex tool-use trajectories. Third, we introduce Progressive Preference Refinement (PPR) to resolve fine-grained preference ambiguities. By mining hard negatives from ranking violations and applying bidirectional preference alignment, PPR minimizes the convex upper bound of pairwise ranking errors. Experiments on benchmarks confirm that AgenticRec significantly outperforms baselines, validating the necessity of unifying reasoning, tool use, and ranking optimization.