Telegrapher's Generative Model via Kac Flows

TL;DR

This paper introduces a novel flow-based generative model based on the telegrapher's equation, offering advantages over diffusion models through a bounded velocity norm and scalable sampling.

Contribution

It proposes a new Kac flow model derived from the telegrapher's equation, extending to multi-dimensional processes and demonstrating improved scalability and performance.

Findings

Kac flow has a globally bounded velocity norm.

The model converges to diffusion in the asymptotic limit.

Numerical experiments show advantages over diffusion models.

Abstract

We break the mold in flow-based generative modeling by proposing a new model based on the damped wave equation, also known as telegrapher's equation. Similar to the diffusion equation and Brownian motion, there is a Feynman-Kac type relation between the telegrapher's equation and the stochastic Kac process in 1D. The Kac flow evolves stepwise linearly in time, so that the probability flow is Lipschitz continuous in the Wasserstein distance and, in contrast to diffusion flows, the norm of the velocity is globally bounded. Furthermore, the Kac model has the diffusion model as its asymptotic limit. We extend these considerations to a multi-dimensional stochastic process which consists of independent 1D Kac processes in each spatial component. We show that this process gives rise to an absolutely continuous curve in the Wasserstein space and compute the conditional velocity field starting in a Dirac point analytically. Using the framework of flow matching, we train a neural network that approximates the velocity field and use it for sample generation. Our numerical experiments demonstrate the scalability of our approach, and show its advantages over diffusion models.