Taming Rectified Flow for Inversion and Editing

TL;DR

This paper introduces RF-Solver and RF-Edit, novel methods that improve inversion accuracy and editing quality in rectified-flow-based diffusion models for images and videos without additional training.

Contribution

We propose RF-Solver, a training-free sampler that enhances inversion precision, and RF-Edit, a feature-sharing framework for high-quality image and video editing.

Findings

RF-Solver significantly reduces inversion errors.

RF-Edit preserves structural information during editing.

Our methods outperform existing approaches in various tasks.

Abstract

Rectified-flow-based diffusion transformers like FLUX and OpenSora have demonstrated outstanding performance in the field of image and video generation. Despite their robust generative capabilities, these models often struggle with inversion inaccuracies, which could further limit their effectiveness in downstream tasks such as image and video editing. To address this issue, we propose RF-Solver, a novel training-free sampler that effectively enhances inversion precision by mitigating the errors in the ODE-solving process of rectified flow. Specifically, we derive the exact formulation of the rectified flow ODE and apply the high-order Taylor expansion to estimate its nonlinear components, significantly enhancing the precision of ODE solutions at each timestep. Building upon RF-Solver, we further propose RF-Edit, a general feature-sharing-based framework for image and video editing. By incorporating self-attention features from the inversion process into the editing process, RF-Edit effectively preserves the structural information of the source image or video while achieving high-quality editing results. Our approach is compatible with any pre-trained rectified-flow-based models for image and video tasks, requiring no additional training or optimization. Extensive experiments across generation, inversion, and editing tasks in both image and video modalities demonstrate the superiority and versatility of our method. The source code is available at https://github.com/wangjiangshan0725/RF-Solver-Edit.