Efficient CDF Approximations for Normalizing Flows

TL;DR

This paper introduces a novel method for approximating the CDF in normalizing flows using boundary flux estimation, significantly improving sample efficiency over traditional Monte Carlo methods.

Contribution

It leverages the divergence theorem and flow properties to develop deterministic and stochastic CDF estimators, enhancing efficiency and accuracy.

Findings

Deterministic estimator improves boundary subdivision accuracy.

Stochastic estimator provides unbiased CDF estimates.

Experiments show better sample efficiency on benchmarks.

Abstract

Normalizing flows model a complex target distribution in terms of a bijective transform operating on a simple base distribution. As such, they enable tractable computation of a number of important statistical quantities, particularly likelihoods and samples. Despite these appealing properties, the computation of more complex inference tasks, such as the cumulative distribution function (CDF) over a complex region (e.g., a polytope) remains challenging. Traditional CDF approximations using Monte-Carlo techniques are unbiased but have unbounded variance and low sample efficiency. Instead, we build upon the diffeomorphic properties of normalizing flows and leverage the divergence theorem to estimate the CDF over a closed region in target space in terms of the flux across its \emph{boundary}, as induced by the normalizing flow. We describe both deterministic and stochastic instances of this estimator: while the deterministic variant iteratively improves the estimate by strategically subdividing the boundary, the stochastic variant provides unbiased estimates. Our experiments on popular flow architectures and UCI benchmark datasets show a marked improvement in sample efficiency as compared to traditional estimators.