Constructive conditional normalizing flows

TL;DR

This paper introduces a novel method for approximating conditional transformations and their pushforwards using neural network-based flows, with explicit constructions and probabilistic alternatives for regular maps.

Contribution

It provides explicit and probabilistic constructions for flow-based approximations of diffeomorphisms and their pushforwards, leveraging a polar-like decomposition and neural network architectures.

Findings

Explicit construction for flow approximation of diffeomorphisms.

Probabilistic method for regular maps with scalable discontinuities.

Implementation of flow components via convex functions and shear flows.

Abstract

Motivated by applications in conditional sampling, given a probability measure $\mu$ and a diffeomorphism $\phi$, we consider the problem of simultaneously approximating $\phi$ and the pushforward $\phi_{\#}\mu$ by means of the flow of a continuity equation whose velocity field is a perceptron neural network with piecewise constant weights. We provide an explicit construction based on a polar-like decomposition of the Lagrange interpolant of $\phi$. The latter involves a compressible component, given by the gradient of a particular convex function, which can be realized exactly, and an incompressible component, which -- after approximating via permutations -- can be implemented through shear flows intrinsic to the continuity equation. For more regular maps $\phi$ -- such as the Kn\"othe-Rosenblatt rearrangement -- we provide an alternative, probabilistic construction inspired by the Maurey empirical method, in which the number of discontinuities in the weights doesn't scale inversely with the ambient dimension.