PreferThinker: Reasoning-based Personalized Image Preference Assessment

TL;DR

This paper introduces PreferThinker, a reasoning-based framework for personalized image preference assessment that predicts user profiles from limited references and provides interpretable multi-dimensional evaluations, leveraging large-scale data and structured reasoning.

Contribution

It proposes a novel predict-then-assess paradigm with a preference profile predictor, a large-scale CoT-style dataset, and a two-stage training strategy including reinforcement learning.

Findings

Outperforms existing methods in personalized preference assessment.

Effectively predicts user preference profiles from limited references.

Provides interpretable, multi-dimensional image assessments.

Abstract

Personalized image preference assessment aims to evaluate an individual user's image preferences by relying only on a small set of reference images as prior information. Existing methods mainly focus on general preference assessment, training models with large-scale data to tackle well-defined tasks such as text-image alignment. However, these approaches struggle to handle personalized preference because user-specific data are scarce and not easily scalable, and individual tastes are often diverse and complex. To overcome these challenges, we introduce a common preference profile that serves as a bridge across users, allowing large-scale user data to be leveraged for training profile prediction and capturing complex personalized preferences. Building on this idea, we propose a reasoning-based personalized image preference assessment framework that follows a \textit{predict-then-assess} paradigm: it first predicts a user's preference profile from reference images, and then provides interpretable, multi-dimensional scores and assessments of candidate images based on the predicted profile. To support this, we first construct a large-scale Chain-of-Thought (CoT)-style personalized assessment dataset annotated with diverse user preference profiles and high-quality CoT-style reasoning, enabling explicit supervision of structured reasoning. Next, we adopt a two-stage training strategy: a cold-start supervised fine-tuning phase to empower the model with structured reasoning capabilities, followed by reinforcement learning to incentivize the model to explore more reasonable assessment paths and enhance generalization. Furthermore, we propose a similarity-aware prediction reward to encourage better prediction of the user's preference profile, which facilitates more reasonable assessments exploration. Extensive experiments demonstrate the superiority of the proposed method.