Diff2Flow: Training Flow Matching Models via Diffusion Model Alignment

TL;DR

Diff2Flow introduces a novel method to efficiently transfer knowledge from diffusion models to flow matching models, enabling faster inference and improved performance across tasks without extra computational costs.

Contribution

The paper presents Diff2Flow, a framework that aligns diffusion models with flow matching, allowing efficient finetuning and superior performance in generative tasks.

Findings

Outperforms naive FM and diffusion finetuning methods.

Achieves superior or competitive results across various tasks.

Enables parameter-efficient finetuning without extra computation.

Abstract

Diffusion models have revolutionized generative tasks through high-fidelity outputs, yet flow matching (FM) offers faster inference and empirical performance gains. However, current foundation FM models are computationally prohibitive for finetuning, while diffusion models like Stable Diffusion benefit from efficient architectures and ecosystem support. This work addresses the critical challenge of efficiently transferring knowledge from pre-trained diffusion models to flow matching. We propose Diff2Flow, a novel framework that systematically bridges diffusion and FM paradigms by rescaling timesteps, aligning interpolants, and deriving FM-compatible velocity fields from diffusion predictions. This alignment enables direct and efficient FM finetuning of diffusion priors with no extra computation overhead. Our experiments demonstrate that Diff2Flow outperforms na\"ive FM and diffusion finetuning particularly under parameter-efficient constraints, while achieving superior or competitive performance across diverse downstream tasks compared to state-of-the-art methods. We will release our code at https://github.com/CompVis/diff2flow.