Tuning-Free Inversion-Enhanced Control for Consistent Image Editing

TL;DR

This paper introduces TIC, a tuning-free method that enhances real image editing by improving reconstruction accuracy and content consistency through inversion features and mask-guided attention, outperforming previous approaches.

Contribution

The paper proposes a novel tuning-free inversion-enhanced control method that correlates inversion and sampling features for consistent image editing without fine-tuning.

Findings

Outperforms previous methods in reconstruction quality

Achieves more content-consistent editing results

Effective in various real-world editing scenarios

Abstract

Consistent editing of real images is a challenging task, as it requires performing non-rigid edits (e.g., changing postures) to the main objects in the input image without changing their identity or attributes. To guarantee consistent attributes, some existing methods fine-tune the entire model or the textual embedding for structural consistency, but they are time-consuming and fail to perform non-rigid edits. Other works are tuning-free, but their performances are weakened by the quality of Denoising Diffusion Implicit Model (DDIM) reconstruction, which often fails in real-world scenarios. In this paper, we present a novel approach called Tuning-free Inversion-enhanced Control (TIC), which directly correlates features from the inversion process with those from the sampling process to mitigate the inconsistency in DDIM reconstruction. Specifically, our method effectively obtains inversion features from the key and value features in the self-attention layers, and enhances the sampling process by these inversion features, thus achieving accurate reconstruction and content-consistent editing. To extend the applicability of our method to general editing scenarios, we also propose a mask-guided attention concatenation strategy that combines contents from both the inversion and the naive DDIM editing processes. Experiments show that the proposed method outperforms previous works in reconstruction and consistent editing, and produces impressive results in various settings.