Neural Spline Flows

TL;DR

Neural Spline Flows introduce a fully differentiable, invertible module using monotonic rational-quadratic splines to enhance the flexibility of normalizing flows for density estimation and generative modeling.

Contribution

The paper presents a novel neural spline flow module that improves the expressiveness of normalizing flows while maintaining invertibility and analytic tractability.

Findings

Enhanced density estimation performance

Improved variational inference results

Better generative modeling of images

Abstract

A normalizing flow models a complex probability density as an invertible transformation of a simple base density. Flows based on either coupling or autoregressive transforms both offer exact density evaluation and sampling, but rely on the parameterization of an easily invertible elementwise transformation, whose choice determines the flexibility of these models. Building upon recent work, we propose a fully-differentiable module based on monotonic rational-quadratic splines, which enhances the flexibility of both coupling and autoregressive transforms while retaining analytic invertibility. We demonstrate that neural spline flows improve density estimation, variational inference, and generative modeling of images.