Block Neural Autoregressive Flow

TL;DR

This paper introduces Block Neural Autoregressive Flow (B-NAF), a more compact and efficient neural autoregressive flow model that maintains universal approximation capabilities with significantly fewer parameters.

Contribution

B-NAF models a bijection directly with a single feed-forward network, ensuring invertibility through block matrices, reducing parameter count compared to previous methods.

Findings

B-NAF is competitive in density estimation tasks.

B-NAF uses orders of magnitude fewer parameters.

B-NAF performs well in approximate inference for latent variable models.

Abstract

Normalising flows (NFS) map two density functions via a differentiable bijection whose Jacobian determinant can be computed efficiently. Recently, as an alternative to hand-crafted bijections, Huang et al. (2018) proposed neural autoregressive flow (NAF) which is a universal approximator for density functions. Their flow is a neural network (NN) whose parameters are predicted by another NN. The latter grows quadratically with the size of the former and thus an efficient technique for parametrization is needed. We propose block neural autoregressive flow (B-NAF), a much more compact universal approximator of density functions, where we model a bijection directly using a single feed-forward network. Invertibility is ensured by carefully designing each affine transformation with block matrices that make the flow autoregressive and (strictly) monotone. We compare B-NAF to NAF and other established flows on density estimation and approximate inference for latent variable models. Our proposed flow is competitive across datasets while using orders of magnitude fewer parameters.