On The Hidden Biases of Flow Matching Samplers

TL;DR

This paper analyzes the biases in flow matching samplers by examining finite-sample effects, revealing how empirical estimators and sample surrogates influence the statistical target and particle dynamics.

Contribution

It introduces a hierarchical empirical model for flow matching, derives exact minimizers, and clarifies finite-sample biases and their impact on sampler behavior.

Findings

Replacing the target law with a finite-sample surrogate alters the statistical target.

Empirical minimizers are generally not gradient fields, even with conditional flows.

Gaussian bases yield exponential tail bounds for kinetic energies.

Abstract

Flow matching (FM) constructs continuous-time ODE samplers by prescribing probability paths between a base distribution and a target distribution. In this note, we study FM through the lens of finite-sample plug-in estimation. In addition to replacing population expectations by sample averages, one may replace the target distribution itself by a finite-sample surrogate, ranging from the empirical measure to a smoothed estimator. This viewpoint yields a natural hierarchy of empirical FM models. For affine conditional flows, we derive the exact empirical minimizer and identify a smoothed plug-in regime in which the terminal law is exactly a kernel-mixture estimator. This plug-in perspective clarifies several coupled finite-sample biases of empirical FM. First, replacing the target law by a finite-sample surrogate changes the statistical target. Second, the empirical minimizer is generally not a gradient field, even when each conditional flow is. Third, a fixed empirical marginal path does not determine a unique particle dynamics: one may add extra vector fields whose probability flux has zero divergence without changing the marginal path. For Gaussian affine conditional paths, we give explicit families of such flux-null corrections. Finally, the source distribution provides a primary mechanism controlling upper tails of kinetic energy. In particular, Gaussian bases yield exponential upper-tail bounds for instantaneous and integrated kinetic energies, whereas polynomially tailed bases yield corresponding polynomial upper-tail bounds.