ImgEdit: A Unified Image Editing Dataset and Benchmark

TL;DR

ImgEdit introduces a large, high-quality dataset and benchmark for image editing, enabling significant improvements in open-source image editing models through comprehensive data and evaluation tools.

Contribution

The paper presents ImgEdit, a new extensive dataset and benchmark for image editing, along with a novel editing model, ImgEdit-E1, and a comprehensive evaluation suite.

Findings

ImgEdit dataset surpasses existing datasets in quality and task diversity.

ImgEdit-E1 outperforms existing open-source models on multiple editing tasks.

The benchmark provides detailed insights into current image editing model capabilities.

Abstract

Recent advancements in generative models have enabled high-fidelity text-to-image generation. However, open-source image-editing models still lag behind their proprietary counterparts, primarily due to limited high-quality data and insufficient benchmarks. To overcome these limitations, we introduce ImgEdit, a large-scale, high-quality image-editing dataset comprising 1.2 million carefully curated edit pairs, which contain both novel and complex single-turn edits, as well as challenging multi-turn tasks. To ensure the data quality, we employ a multi-stage pipeline that integrates a cutting-edge vision-language model, a detection model, a segmentation model, alongside task-specific in-painting procedures and strict post-processing. ImgEdit surpasses existing datasets in both task novelty and data quality. Using ImgEdit, we train ImgEdit-E1, an editing model using Vision Language Model to process the reference image and editing prompt, which outperforms existing open-source models on multiple tasks, highlighting the value of ImgEdit and model design. For comprehensive evaluation, we introduce ImgEdit-Bench, a benchmark designed to evaluate image editing performance in terms of instruction adherence, editing quality, and detail preservation. It includes a basic testsuite, a challenging single-turn suite, and a dedicated multi-turn suite. We evaluate both open-source and proprietary models, as well as ImgEdit-E1, providing deep analysis and actionable insights into the current behavior of image-editing models. The source data are publicly available on https://github.com/PKU-YuanGroup/ImgEdit.