Principal Manifold Flows

TL;DR

This paper introduces principal manifold flows (PF), a new class of normalizing flows that leverage geometric principal manifolds for improved density estimation on complex, variable-dimensional data manifolds.

Contribution

The paper proposes principal manifold flows (PF) and an efficient variant (iPF), integrating geometric structures into normalizing flows for enhanced density estimation and manifold learning.

Findings

PFs can learn principal manifolds across diverse datasets.

PFs perform density estimation on manifolds with variable dimensions.

iPF offers a more efficient training process than regular injective flows.

Abstract

Normalizing flows map an independent set of latent variables to their samples using a bijective transformation. Despite the exact correspondence between samples and latent variables, their high level relationship is not well understood. In this paper we characterize the geometric structure of flows using principal manifolds and understand the relationship between latent variables and samples using contours. We introduce a novel class of normalizing flows, called principal manifold flows (PF), whose contours are its principal manifolds, and a variant for injective flows (iPF) that is more efficient to train than regular injective flows. PFs can be constructed using any flow architecture, are trained with a regularized maximum likelihood objective and can perform density estimation on all of their principal manifolds. In our experiments we show that PFs and iPFs are able to learn the principal manifolds over a variety of datasets. Additionally, we show that PFs can perform density estimation on data that lie on a manifold with variable dimensionality, which is not possible with existing normalizing flows.