On the Closed-Form of Flow Matching: Generalization Does Not Arise from Target Stochasticity

TL;DR

This paper investigates the role of stochasticity in flow matching models, showing that the stochastic and closed-form loss variants perform similarly, with the latter sometimes outperforming the former in high-dimensional image generation tasks.

Contribution

It demonstrates that the stochastic nature of the flow matching loss is not essential for generalization, and provides a closed-form solution that can enhance model performance.

Findings

Stochastic and closed-form flow matching losses are nearly equivalent in high-dimensional settings.

Closed-form flow matching can outperform stochastic variants in image datasets.

Stochasticity is not a key factor for generalization in flow matching models.

Abstract

Modern deep generative models can now produce high-quality synthetic samples that are often indistinguishable from real training data. A growing body of research aims to understand why recent methods, such as diffusion and flow matching techniques, generalize so effectively. Among the proposed explanations are the inductive biases of deep learning architectures and the stochastic nature of the conditional flow matching loss. In this work, we rule out the noisy nature of the loss as a key factor driving generalization in flow matching. First, we empirically show that in high-dimensional settings, the stochastic and closed-form versions of the flow matching loss yield nearly equivalent losses. Then, using state-of-the-art flow matching models on standard image datasets, we demonstrate that both variants achieve comparable statistical performance, with the surprising observation that using the closed-form can even improve performance.