PROMISE: Process Reward Models Unlock Test-Time Scaling Laws in Generative Recommendations

TL;DR

Promise introduces a framework that integrates step-by-step verification into generative recommendation models, reducing semantic errors and enabling smaller models to perform as well as larger ones through test-time scaling.

Contribution

It proposes a novel PRM-based approach with dense verification and guided search, unlocking test-time scaling laws for recommender systems.

Findings

Reduces Semantic Drift in generative recommendations

Enables smaller models to match larger models' performance with increased inference compute

Improves recommendation accuracy significantly in large-scale online tests

Abstract

Generative Recommendation has emerged as a promising paradigm, reformulating recommendation as a sequence-to-sequence generation task over hierarchical Semantic IDs. However, existing methods suffer from a critical issue we term Semantic Drift, where errors in early, high-level tokens irreversibly divert the generation trajectory into irrelevant semantic subspaces. Inspired by Process Reward Models (PRMs) that enhance reasoning in Large Language Models, we propose Promise, a novel framework that integrates dense, step-by-step verification into generative models. Promise features a lightweight PRM to assess the quality of intermediate inference steps, coupled with a PRM-guided Beam Search strategy that leverages dense feedback to dynamically prune erroneous branches. Crucially, our approach unlocks Test-Time Scaling Laws for recommender systems: by increasing inference compute, smaller models can match or surpass larger models. Extensive offline experiments and online A/B tests on a large-scale platform demonstrate that Promise effectively mitigates Semantic Drift, significantly improving recommendation accuracy while enabling efficient deployment.