Iterative Multi-granular Image Editing using Diffusion Models

TL;DR

This paper introduces EMILIE, a diffusion-based image editing framework enabling iterative, multi-granular modifications with spatial control, addressing limitations of existing one-shot editing models.

Contribution

The paper proposes EMILIE, a novel latent iteration strategy and gradient control for iterative, multi-granular image editing using diffusion models, and introduces a new benchmark dataset.

Findings

EMILIE outperforms recent state-of-the-art methods in iterative multi-granular editing.

The latent iteration strategy effectively enables iterative refinement of images.

Gradient control allows flexible spatial scope of edits.

Abstract

Recent advances in text-guided image synthesis has dramatically changed how creative professionals generate artistic and aesthetically pleasing visual assets. To fully support such creative endeavors, the process should possess the ability to: 1) iteratively edit the generations and 2) control the spatial reach of desired changes (global, local or anything in between). We formalize this pragmatic problem setting as Iterative Multi-granular Editing. While there has been substantial progress with diffusion-based models for image synthesis and editing, they are all one shot (i.e., no iterative editing capabilities) and do not naturally yield multi-granular control (i.e., covering the full spectrum of local-to-global edits). To overcome these drawbacks, we propose EMILIE: Iterative Multi-granular Image Editor. EMILIE introduces a novel latent iteration strategy, which re-purposes a pre-trained diffusion model to facilitate iterative editing. This is complemented by a gradient control operation for multi-granular control. We introduce a new benchmark dataset to evaluate our newly proposed setting. We conduct exhaustive quantitatively and qualitatively evaluation against recent state-of-the-art approaches adapted to our task, to being out the mettle of EMILIE. We hope our work would attract attention to this newly identified, pragmatic problem setting.