PostEdit: Posterior Sampling for Efficient Zero-Shot Image Editing

TL;DR

PostEdit is a novel image editing method that combines posterior sampling with diffusion models to achieve high efficiency, controllability, and background preservation without requiring inversion or training, outperforming existing methods.

Contribution

The paper introduces PostEdit, a new diffusion-based image editing approach that efficiently balances editing quality, background consistency, and speed without inversion or training.

Findings

Achieves state-of-the-art editing performance.

Preserves unedited regions accurately.

Operates in approximately 1.5 seconds with 18 GB GPU memory.

Abstract

In the field of image editing, three core challenges persist: controllability, background preservation, and efficiency. Inversion-based methods rely on time-consuming optimization to preserve the features of the initial images, which results in low efficiency due to the requirement for extensive network inference. Conversely, inversion-free methods lack theoretical support for background similarity, as they circumvent the issue of maintaining initial features to achieve efficiency. As a consequence, none of these methods can achieve both high efficiency and background consistency. To tackle the challenges and the aforementioned disadvantages, we introduce PostEdit, a method that incorporates a posterior scheme to govern the diffusion sampling process. Specifically, a corresponding measurement term related to both the initial features and Langevin dynamics is introduced to optimize the estimated image generated by the given target prompt. Extensive experimental results indicate that the proposed PostEdit achieves state-of-the-art editing performance while accurately preserving unedited regions. Furthermore, the method is both inversion- and training-free, necessitating approximately 1.5 seconds and 18 GB of GPU memory to generate high-quality results.