Learning User Preferences for Image Generation Model

TL;DR

This paper introduces a novel approach using multimodal large language models with contrastive preference loss and preference tokens to improve personalized image generation by accurately capturing individual user tastes.

Contribution

It presents a new method that models personalized user preferences through contrastive learning and preference tokens, addressing limitations of static and general preference assumptions.

Findings

Outperforms existing methods in preference prediction accuracy

Effectively identifies users with similar aesthetic tastes

Provides more precise image generation aligned with individual preferences

Abstract

User preference prediction requires a comprehensive and accurate understanding of individual tastes. This includes both surface-level attributes, such as color and style, and deeper content-related aspects, such as themes and composition. However, existing methods typically rely on general human preferences or assume static user profiles, often neglecting individual variability and the dynamic, multifaceted nature of personal taste. To address these limitations, we propose an approach built upon Multimodal Large Language Models, introducing contrastive preference loss and preference tokens to learn personalized user preferences from historical interactions. The contrastive preference loss is designed to effectively distinguish between user ''likes'' and ''dislikes'', while the learnable preference tokens capture shared interest representations among existing users, enabling the model to activate group-specific preferences and enhance consistency across similar users. Extensive experiments demonstrate our model outperforms other methods in preference prediction accuracy, effectively identifying users with similar aesthetic inclinations and providing more precise guidance for generating images that align with individual tastes. The project page is \texttt{https://learn-user-pref.github.io/}.