PSR: Scaling Multi-Subject Personalized Image Generation with Pairwise Subject-Consistency Rewards

TL;DR

This paper introduces a scalable pipeline and reward-based training strategy to improve multi-subject personalized image generation, addressing previous limitations in subject consistency and prompt adherence.

Contribution

It presents a novel multi-subject data generation pipeline and Pairwise Subject-Consistency Rewards to enhance multi-subject personalization models.

Findings

Improved subject consistency in multi-subject image generation.

Enhanced adherence to textual prompts in generated images.

Established a new benchmark for evaluating multi-subject personalization.

Abstract

Personalized generation models for a single subject have demonstrated remarkable effectiveness, highlighting their significant potential. However, when extended to multiple subjects, existing models often exhibit degraded performance, particularly in maintaining subject consistency and adhering to textual prompts. We attribute these limitations to the absence of high-quality multi-subject datasets and refined post-training strategies. To address these challenges, we propose a scalable multi-subject data generation pipeline that leverages powerful single-subject generation models to construct diverse and high-quality multi-subject training data. Through this dataset, we first enable single-subject personalization models to acquire knowledge of synthesizing multi-image and multi-subject scenarios. Furthermore, to enhance both subject consistency and text controllability, we design a set of Pairwise Subject-Consistency Rewards and general-purpose rewards, which are incorporated into a refined reinforcement learning stage. To comprehensively evaluate multi-subject personalization, we introduce a new benchmark that assesses model performance using seven subsets across three dimensions. Extensive experiments demonstrate the effectiveness of our approach in advancing multi-subject personalized image generation. Github Link: https://github.com/wang-shulei/PSR