On the continuity of flows

TL;DR

This paper analyzes how topological mismatches between prior and target distributions in continuous normalizing flows can cause discontinuities in the optimal velocity fields, impacting flow matching performance.

Contribution

It provides a theoretical analysis demonstrating the emergence of discontinuities due to topological constraints and validates these findings empirically.

Findings

Optimal velocity fields exhibit jump discontinuities at decision boundaries.

Discontinuities become more pronounced as the distribution approaches the target.

Topological mismatch, not just loss function choice, causes these discontinuities.

Abstract

Flow matching has emerged as a powerful framework for generative modeling through continuous normalizing flows. We investigate a potential topological constraint: when the prior distribution and target distribution have mismatched topology (e.g., unimodal to multimodal), the optimal velocity field under standard flow matching objectives may exhibit spatial discontinuities. We suggest that this discontinuity arises from the requirement that continuous flows must bifurcate to map a single mode to multiple modes, forcing particles to make discrete routing decisions at intermediate times. Through theoretical analysis on bimodal Gaussian mixtures, we demonstrate that the optimal velocity field exhibits jump discontinuities along decision boundaries, with magnitude approaching infinity as time approaches the target distribution. Our analysis suggests that this phenomenon is not specific to $L^2$ loss, but rather may be a consequence of topological mismatch between distributions. We validate our theory empirically and discuss potential implications for flow matching on manifolds, connecting our findings to recent work on Riemannian flow matching and the challenge of learning discontinuous representations in neural networks.