Neural Inverse Transform Sampler

TL;DR

The paper introduces NITS, a neural network-based framework that efficiently models and samples from high-dimensional probability densities, enabling fast, exact likelihood evaluation and competitive generative modeling performance.

Contribution

It presents a novel neural inverse transform sampling method that extends to high dimensions, allowing for expressive, differentiable density estimation with fast sampling and likelihood computation.

Findings

NITS achieves state-of-the-art results on CIFAR-10.

NITS provides fast, exact likelihood evaluation.

NITS effectively models high-dimensional densities.

Abstract

Any explicit functional representation $f$ of a density is hampered by two main obstacles when we wish to use it as a generative model: designing $f$ so that sampling is fast, and estimating $Z = \int f$ so that $Z^{-1}f$ integrates to 1. This becomes increasingly complicated as $f$ itself becomes complicated. In this paper, we show that when modeling one-dimensional conditional densities with a neural network, $Z$ can be exactly and efficiently computed by letting the network represent the cumulative distribution function of a target density, and applying a generalized fundamental theorem of calculus. We also derive a fast algorithm for sampling from the resulting representation by the inverse transform method. By extending these principles to higher dimensions, we introduce the \textbf{Neural Inverse Transform Sampler (NITS)}, a novel deep learning framework for modeling and sampling from general, multidimensional, compactly-supported probability densities. NITS is a highly expressive density estimator that boasts end-to-end differentiability, fast sampling, and exact and cheap likelihood evaluation. We demonstrate the applicability of NITS by applying it to realistic, high-dimensional density estimation tasks: likelihood-based generative modeling on the CIFAR-10 dataset, and density estimation on the UCI suite of benchmark datasets, where NITS produces compelling results rivaling or surpassing the state of the art.