Exploring Iterative Manifold Constraint for Zero-shot Image Editing

TL;DR

This paper introduces ZZEdit, a zero-shot image editing method that uses an intermediate-inverted latent and iterative manifold constraints to improve editability and fidelity over traditional inversion-then-editing approaches.

Contribution

The paper proposes a novel zero-shot editing paradigm with an intermediate-inverted latent and a ZigZag denoising process, reducing fidelity errors and enhancing editing quality.

Findings

ZZEdit outperforms existing methods in editability and fidelity.

Iterative manifold constraint reduces fidelity errors.

Intermediate-inverted latents improve trade-off between editability and fidelity.

Abstract

Editability and fidelity are two essential demands for text-driven image editing, which expects that the editing area should align with the target prompt and the rest remain unchanged separately. The current cutting-edge editing methods usually obey an "inversion-then-editing" pipeline, where the input image is inverted to an approximate Gaussian noise ${z}_T$, based on which a sampling process is conducted using the target prompt. Nevertheless, we argue that it is not a good choice to use a near-Gaussian noise as a pivot for further editing since it would bring plentiful fidelity errors. We verify this by a pilot analysis, discovering that intermediate-inverted latents can achieve a better trade-off between editability and fidelity than the fully-inverted ${z}_T$. Based on this, we propose a novel zero-shot editing paradigm dubbed ZZEdit, which first locates a qualified intermediate-inverted latent marked as ${z}_p$ as a better editing pivot, which is sufficient-for-editing while structure-preserving. Then, a ZigZag process is designed to execute denoising and inversion alternately, which progressively inject target guidance to ${z}_p$ while preserving the structure information of $p$ step. Afterwards, to achieve the same step number of inversion and denoising, we execute a pure sampling process under the target prompt. Essentially, our ZZEdit performs iterative manifold constraint between the manifold of $M_{p}$ and $M_{p-1}$, leading to fewer fidelity errors. Extensive experiments highlight the effectiveness of ZZEdit in diverse image editing scenarios compared with the "inversion-then-editing" pipeline.