Continuous normalizing flows on manifolds

TL;DR

This paper extends continuous normalizing flows to arbitrary smooth manifolds using differential geometry, enabling reparameterizable sampling from complex distributions on diverse geometric spaces.

Contribution

It introduces a general methodology for parameterizing vector fields on manifolds and demonstrates scalable gradient-based learning in this setting.

Findings

Successfully applied to various geometric spaces

Achieved reparameterizable sampling from complex distributions

Provided a scalable unbiased divergence estimator

Abstract

Normalizing flows are a powerful technique for obtaining reparameterizable samples from complex multimodal distributions. Unfortunately, current approaches are only available for the most basic geometries and fall short when the underlying space has a nontrivial topology, limiting their applicability for most real-world data. Using fundamental ideas from differential geometry and geometric control theory, we describe how the recently introduced Neural ODEs and continuous normalizing flows can be extended to arbitrary smooth manifolds. We propose a general methodology for parameterizing vector fields on these spaces and demonstrate how gradient-based learning can be performed. Additionally, we provide a scalable unbiased estimator for the divergence in this generalized setting. Experiments on a diverse selection of spaces empirically showcase the defined framework's ability to obtain reparameterizable samples from complex distributions.