Flow Matching: Markov Kernels, Stochastic Processes and Transport Plans

TL;DR

This paper reviews mathematical techniques for learning velocity fields in flow matching models, highlighting their applications in sampling, Bayesian inverse problems, and connections to other generative methods.

Contribution

It provides a comprehensive mathematical analysis of flow matching, characterizing velocity fields via transport plans, Markov kernels, and stochastic processes, and explores their applications.

Findings

Velocity fields can be characterized using transport plans, Markov kernels, and stochastic processes.

Flow matching techniques are effective for sampling and solving Bayesian inverse problems.

Connections to continuous normalizing flows and score matching are discussed.

Abstract

Among generative neural models, flow matching techniques stand out for their simple applicability and good scaling properties. Here, velocity fields of curves connecting a simple latent and a target distribution are learned. Then the corresponding ordinary differential equation can be used to sample from a target distribution, starting in samples from the latent one. This paper reviews from a mathematical point of view different techniques to learn the velocity fields of absolutely continuous curves in the Wasserstein geometry. We show how the velocity fields can be characterized and learned via i) transport plans (couplings) between latent and target distributions, ii) Markov kernels and iii) stochastic processes, where the latter two include the coupling approach, but are in general broader. Besides this main goal, we show how flow matching can be used for solving Bayesian inverse problems, where the definition of conditional Wasserstein distances plays a central role. Finally, we briefly address continuous normalizing flows and score matching techniques, which approach the learning of velocity fields of curves from other directions.