Training-Free Image Editing with Visual Context Integration and Concept Alignment

TL;DR

VicoEdit is a training-free, inversion-free image editing method that uses visual context and concept alignment to improve consistency and performance over existing approaches.

Contribution

It introduces a novel training-free, inversion-free approach for visual context-aware image editing that outperforms training-based methods.

Findings

Achieves better editing performance than state-of-the-art training-based models.

Eliminates the need for inversion, reducing deviation issues.

Uses concept alignment to enhance editing consistency.

Abstract

In image editing, it is essential to incorporate a context image to convey the user's precise requirements, such as subject appearance or image style. Existing training-based visual context-aware editing methods incur data collection effort and training cost. On the other hand, the training-free alternatives are typically established on diffusion inversion, which struggles with consistency and flexibility. In this work, we propose VicoEdit, a training-free and inversion-free method to inject the visual context into the pretrained text-prompted editing model. More specifically, VicoEdit directly transforms the source image into the target one based on the visual context, thereby eliminating the need for inversion that can lead to deviated trajectories. Moreover, we design a posterior sampling approach guided by concept alignment to enhance the editing consistency. Empirical results demonstrate that our training-free method achieves even better editing performance than the state-of-the-art training-based models.