RecRM-Bench: Benchmarking Multidimensional Reward Modeling for Agentic Recommender Systems

TL;DR

RecRM-Bench is a comprehensive benchmark dataset designed to evaluate multi-dimensional reward models for agentic recommender systems, addressing the limitations of single-dimensional reward approaches.

Contribution

It introduces the largest dataset for multi-dimensional reward modeling in recommender systems and proposes a systematic framework for reward model construction.

Findings

Over 1 million entries across four evaluation dimensions.

Supports assessment from syntax to complex intent grounding.

Provides a foundation for training advanced reward models.

Abstract

The integration of Large Language Model (LLM) agents is transforming recommender systems from simple query-item matching towards deeply personalized and interactive recommendations. Reinforcement Learning (RL) provides an essential framework for the optimization of these agents in recommendation tasks. However, current methodologies remain limited by a reliance on single dimensional outcome-based rewards that focus exclusively on final user interactions, overlooking critical intermediate capabilities, such as instruction following and complex intent understanding. Despite the necessity for designing multi-dimensional reward, the field lacks a standardized benchmark to facilitate this development. To bridge this gap, we introduce RecRM-Bench, the largest and most comprehensive benchmark to date for agentic recommender systems. It comprises over 1 million structured entries across four core evaluation dimensions: instruction following, factual consistency, query-item relevance, and fine-grained user behavior prediction. By supporting comprehensive assessment from syntactic compliance to complex intent grounding and preference modeling, RecRM-Bench provides a foundational dataset for training sophisticated reward models. Furthermore, we propose a systematic framework for the construction of multi-dimensional reward models and the integration of a hybrid reward function, establishing a robust foundation for developing reliable and highly capable agentic recommender systems. The complete RecRM-Bench dataset is publicly available at https://huggingface.co/datasets/wwzeng/RecRM-Bench.