Funnels: Exact maximum likelihood with dimensionality reduction

TL;DR

This paper introduces the funnel layer within normalizing flows, enabling dimension reduction while maintaining likelihood-based training, leading to improved or comparable performance with smaller latent spaces.

Contribution

It proposes a novel funnel layer for surjective flows that reduces dimensionality, expanding the capabilities of normalizing flows with diverse transformations.

Findings

Funnel flows perform as well as or better than existing models.

The funnel layer effectively reduces latent space size.

Applicable with various transformation types like convolution and feedforward.

Abstract

Normalizing flows are diffeomorphic, typically dimension-preserving, models trained using the likelihood of the model. We use the SurVAE framework to construct dimension reducing surjective flows via a new layer, known as the funnel. We demonstrate its efficacy on a variety of datasets, and show it improves upon or matches the performance of existing flows while having a reduced latent space size. The funnel layer can be constructed from a wide range of transformations including restricted convolution and feed forward layers.