Personalized Reward Modeling for Text-to-Image Generation

TL;DR

This paper introduces PIGReward, a personalized evaluation model for text-to-image generation that uses reasoning and limited user data to better align images with individual preferences.

Contribution

PIGReward is a novel personalized reward model that dynamically generates evaluation criteria and uses self-bootstrapping to personalize without user-specific training.

Findings

PIGReward outperforms existing evaluation methods in accuracy.

It provides interpretable, user-specific feedback for image prompt optimization.

Demonstrates scalability and effectiveness in personalized T2I evaluation.

Abstract

Recent text-to-image (T2I) models generate semantically coherent images from textual prompts, yet evaluating how well they align with individual user preferences remains an open challenge. Conventional evaluation methods, general reward functions or similarity-based metrics, fail to capture the diversity and complexity of personal visual tastes. In this work, we present PIGReward, a personalized reward model that dynamically generates user-conditioned evaluation dimensions and assesses images through CoT reasoning. To address the scarcity of user data, PIGReward adopt a self-bootstrapping strategy that reasons over limited reference data to construct rich user contexts, enabling personalization without user-specific training. Beyond evaluation, PIGReward provides personalized feedback that drives user-specific prompt optimization, improving alignment between generated images and individual intent. We further introduce PIGBench, a per-user preference benchmark capturing diverse visual interpretations of shared prompts. Extensive experiments demonstrate that PIGReward surpasses existing methods in both accuracy and interpretability, establishing a scalable and reasoning-based foundation for personalized T2I evaluation and optimization. Taken together, our findings highlight PIGReward as a robust steptoward individually aligned T2I generation.