HP-Edit: A Human-Preference Post-Training Framework for Image Editing

TL;DR

HP-Edit is a post-training framework that uses human preferences and a new dataset to improve image editing models' alignment with human expectations.

Contribution

It introduces HP-Edit, a novel framework utilizing human preference data and a pretrained VLM to enhance diffusion-based image editing.

Findings

Significant improvement in human preference alignment of image editing models.

Development of RealPref-50K, a large-scale human preference dataset.

Introduction of RealPref-Bench for evaluating real-world editing performance.

Abstract

Common image editing tasks typically adopt powerful generative diffusion models as the leading paradigm for real-world content editing. Meanwhile, although reinforcement learning (RL) methods such as Diffusion-DPO and Flow-GRPO have further improved generation quality, efficiently applying Reinforcement Learning from Human Feedback (RLHF) to diffusion-based editing remains largely unexplored, due to a lack of scalable human-preference datasets and frameworks tailored to diverse editing needs. To fill this gap, we propose HP-Edit, a post-training framework for Human Preference-aligned Editing, and introduce RealPref-50K, a real-world dataset across eight common tasks and balancing common object editing. Specifically, HP-Edit leverages a small amount of human-preference scoring data and a pretrained visual large language model (VLM) to develop HP-Scorer--an automatic, human preference-aligned evaluator. We then use HP-Scorer both to efficiently build a scalable preference dataset and to serve as the reward function for post-training the editing model. We also introduce RealPref-Bench, a benchmark for evaluating real-world editing performance. Extensive experiments demonstrate that our approach significantly enhances models such as Qwen-Image-Edit-2509, aligning their outputs more closely with human preference.