ELF: Exact-Lipschitz Based Universal Density Approximator Flow

TL;DR

This paper introduces ELF, a new flow model with exact Lipschitz constants that is computationally efficient, universal in density approximation, and achieves state-of-the-art results on large datasets.

Contribution

ELF combines residual and autoregressive flow advantages using a simple one-dimensional network with closed-form Lipschitz constants, ensuring efficiency and universality.

Findings

ELF is a universal density approximator.

ELF outperforms existing flows in efficiency and parameter count.

ELF achieves state-of-the-art performance on large-scale datasets.

Abstract

Normalizing flows have grown more popular over the last few years; however, they continue to be computationally expensive, making them difficult to be accepted into the broader machine learning community. In this paper, we introduce a simple one-dimensional one-layer network that has closed form Lipschitz constants; using this, we introduce a new Exact-Lipschitz Flow (ELF) that combines the ease of sampling from residual flows with the strong performance of autoregressive flows. Further, we show that ELF is provably a universal density approximator, more computationally and parameter efficient compared to a multitude of other flows, and achieves state-of-the-art performance on multiple large-scale datasets.