Oscillation Inversion: Understand the structure of Large Flow Model through the Lens of Inversion Method

TL;DR

This paper investigates the oscillatory behavior in inversion methods for large-scale diffusion models, revealing its semantic coherence and leveraging it for effective image editing and enhancement techniques.

Contribution

It provides theoretical insights into oscillatory dynamics in diffusion model inversion and introduces a fast distribution transfer method for various image editing tasks.

Findings

Oscillations are semantically coherent clusters in inversion.

The method improves image enhancement and editing quality.

Quantitative results show effectiveness across multiple tasks.

Abstract

We explore the oscillatory behavior observed in inversion methods applied to large-scale text-to-image diffusion models, with a focus on the "Flux" model. By employing a fixed-point-inspired iterative approach to invert real-world images, we observe that the solution does not achieve convergence, instead oscillating between distinct clusters. Through both toy experiments and real-world diffusion models, we demonstrate that these oscillating clusters exhibit notable semantic coherence. We offer theoretical insights, showing that this behavior arises from oscillatory dynamics in rectified flow models. Building on this understanding, we introduce a simple and fast distribution transfer technique that facilitates image enhancement, stroke-based recoloring, as well as visual prompt-guided image editing. Furthermore, we provide quantitative results demonstrating the effectiveness of our method for tasks such as image enhancement, makeup transfer, reconstruction quality, and guided sampling quality. Higher-quality examples of videos and images are available at \href{https://yanyanzheng96.github.io/oscillation_inversion/}{this link}.