Text-to-Image Rectified Flow as Plug-and-Play Priors

TL;DR

This paper introduces rectified flow models as efficient and high-quality priors for generative tasks, demonstrating their advantages over diffusion models in text-to-3D generation, image inversion, and editing.

Contribution

It presents the first theoretical and experimental validation of rectified flow as a versatile prior, outperforming diffusion models in quality and efficiency.

Findings

Rectified flow priors outperform diffusion-based priors in text-to-3D generation.

The method enables effective image inversion and editing.

Fewer inference steps are needed compared to diffusion models.

Abstract

Large-scale diffusion models have achieved remarkable performance in generative tasks. Beyond their initial training applications, these models have proven their ability to function as versatile plug-and-play priors. For instance, 2D diffusion models can serve as loss functions to optimize 3D implicit models. Rectified flow, a novel class of generative models, enforces a linear progression from the source to the target distribution and has demonstrated superior performance across various domains. Compared to diffusion-based methods, rectified flow approaches surpass in terms of generation quality and efficiency, requiring fewer inference steps. In this work, we present theoretical and experimental evidence demonstrating that rectified flow based methods offer similar functionalities to diffusion models - they can also serve as effective priors. Besides the generative capabilities of diffusion priors, motivated by the unique time-symmetry properties of rectified flow models, a variant of our method can additionally perform image inversion. Experimentally, our rectified flow-based priors outperform their diffusion counterparts - the SDS and VSD losses - in text-to-3D generation. Our method also displays competitive performance in image inversion and editing.