Efficient Flow Matching using Latent Variables

TL;DR

This paper introduces Latent-CFM, a flow matching model that leverages pretrained latent variables to improve training efficiency and generation quality, especially for high-dimensional data and physical processes.

Contribution

It proposes a novel method that incorporates pretrained deep latent variable models into flow matching, enhancing efficiency and interpretability in generative modeling.

Findings

Improved image generation quality with less training and computation.

More physically accurate samples in spatial field generation.

Enables conditional image generation through latent space analysis.

Abstract

Flow matching models have shown great potential in image generation tasks among probabilistic generative models. However, most flow matching models in the literature do not explicitly utilize the underlying clustering structure in the target data when learning the flow from a simple source distribution like the standard Gaussian. This leads to inefficient learning, especially for many high-dimensional real-world datasets, which often reside in a low-dimensional manifold. To this end, we present $\texttt{Latent-CFM}$, which provides efficient training strategies by conditioning on the features extracted from data using pretrained deep latent variable models. Through experiments on synthetic data from multi-modal distributions and widely used image benchmark datasets, we show that $\texttt{Latent-CFM}$ exhibits improved generation quality with significantly less training and computation than state-of-the-art flow matching models by adopting pretrained lightweight latent variable models. Beyond natural images, we consider generative modeling of spatial fields stemming from physical processes. Using a 2d Darcy flow dataset, we demonstrate that our approach generates more physically accurate samples than competing approaches. In addition, through latent space analysis, we demonstrate that our approach can be used for conditional image generation conditioned on latent features, which adds interpretability to the generation process.