Generator Matching: Generative modeling with arbitrary Markov processes

TL;DR

Generator Matching is a versatile framework for generative modeling using arbitrary Markov processes, unifying existing methods and enabling new process designs for improved multimodal data generation.

Contribution

It introduces a unified framework that leverages generator functions for arbitrary Markov processes, expanding the design space and enabling multimodal generative models.

Findings

Unifies diffusion, flow matching, and discrete diffusion models.

Enables construction of multimodal and superposed Markov generative models.

Empirically improves image and multimodal generation performance.

Abstract

We introduce Generator Matching, a modality-agnostic framework for generative modeling using arbitrary Markov processes. Generators characterize the infinitesimal evolution of a Markov process, which we leverage for generative modeling in a similar vein to flow matching: we construct conditional generators which generate single data points, then learn to approximate the marginal generator which generates the full data distribution. We show that Generator Matching unifies various generative modeling methods, including diffusion models, flow matching and discrete diffusion models. Furthermore, it expands the design space to new and unexplored Markov processes such as jump processes. Finally, Generator Matching enables the construction of superpositions of Markov generative models and enables the construction of multimodal models in a rigorous manner. We empirically validate our method on image and multimodal generation, e.g. showing that superposition with a jump process improves performance.