Density estimation on low-dimensional manifolds: an inflation-deflation approach

TL;DR

This paper introduces a novel inflation-deflation approach for normalizing flows that enables density estimation on low-dimensional manifolds, overcoming support limitations without increasing computational complexity.

Contribution

The method inflates data manifolds with noise, trains NFs on this, and deflates the density, allowing universal density estimation on manifolds with known dimensions.

Findings

The approach is exact under certain conditions on the manifold and noise.

It maintains the same computational complexity as standard NFs.

Gaussian noise approximation works well when embedding dimension is large.

Abstract

Normalizing Flows (NFs) are universal density estimators based on Neural Networks. However, this universality is limited: the density's support needs to be diffeomorphic to a Euclidean space. In this paper, we propose a novel method to overcome this limitation without sacrificing universality. The proposed method inflates the data manifold by adding noise in the normal space, trains an NF on this inflated manifold, and, finally, deflates the learned density. Our main result provides sufficient conditions on the manifold and the specific choice of noise under which the corresponding estimator is exact. Our method has the same computational complexity as NFs and does not require computing an inverse flow. We also show that, if the embedding dimension is much larger than the manifold dimension, noise in the normal space can be well approximated by Gaussian noise. This allows using our method for approximating arbitrary densities on unknown manifolds provided that the manifold dimension is known.