The Amazing Stability of Flow Matching

TL;DR

This paper demonstrates that flow-matching models exhibit remarkable stability in sample quality, diversity, and latent representations even when subjected to dataset pruning and architectural changes.

Contribution

The study reveals the robustness of flow-matching models, showing they maintain performance and representations despite significant dataset pruning and configuration variations.

Findings

Flow-matching models remain stable with 50% dataset pruning.

Sample quality and diversity are preserved under pruning.

Latent representations are minimally affected by pruning and architecture changes.

Abstract

The success of deep generative models in generating high-quality and diverse samples is often attributed to particular architectures and large training datasets. In this paper, we investigate the impact of these factors on the quality and diversity of samples generated by \emph{flow-matching} models. Surprisingly, in our experiments on CelebA-HQ dataset, flow matching remains stable even when pruning 50\% of the dataset. That is, the quality and diversity of generated samples are preserved. Moreover, pruning impacts the latent representation only slightly, that is, samples generated by models trained on the full and pruned dataset map to visually similar outputs for a given seed. We observe similar stability when changing the architecture or training configuration, such that the latent representation is maintained under these changes as well. Our results quantify just how strong this stability can be in practice, and help explain the reliability of flow-matching models under various perturbations.