SEED-Data-Edit Technical Report: A Hybrid Dataset for Instructional Image Editing

TL;DR

This paper introduces SEED-Data-Edit, a comprehensive hybrid dataset for instruction-guided image editing, combining automated, real-world, and human-annotated data to improve language-guided image manipulation models.

Contribution

The paper presents a novel hybrid dataset for instruction-guided image editing and demonstrates its effectiveness by fine-tuning a multimodal language model.

Findings

Promising results in instruction-guided image editing tasks

Effective training of language-guided image editing models using SEED-Data-Edit

Dataset supports diverse and iterative image editing scenarios

Abstract

In this technical report, we introduce SEED-Data-Edit: a unique hybrid dataset for instruction-guided image editing, which aims to facilitate image manipulation using open-form language. SEED-Data-Edit is composed of three distinct types of data: (1) High-quality editing data produced by an automated pipeline, ensuring a substantial volume of diverse image editing pairs. (2) Real-world scenario data collected from the internet, which captures the intricacies of user intentions for promoting the practical application of image editing in the real world. (3) High-precision multi-turn editing data annotated by humans, which involves multiple rounds of edits for simulating iterative editing processes. The combination of these diverse data sources makes SEED-Data-Edit a comprehensive and versatile dataset for training language-guided image editing model. We fine-tune a pretrained Multimodal Large Language Model (MLLM) that unifies comprehension and generation with SEED-Data-Edit. The instruction tuned model demonstrates promising results, indicating the potential and effectiveness of SEED-Data-Edit in advancing the field of instructional image editing. The datasets are released in https://huggingface.co/datasets/AILab-CVC/SEED-Data-Edit.