Discretized Approximate Ancestral Sampling

TL;DR

This paper introduces a discretized sampling method for the Fourier Basis Density Model (FBM), enabling efficient and high-quality sampling for band-limited distributions in deep learning applications.

Contribution

It presents a novel discretization-based sampling technique for FBM, with theoretical quality bounds and analysis of tradeoffs compared to existing methods.

Findings

Provides bounds on sampling quality using TV and Wasserstein-1 divergences.

Demonstrates the method's efficiency and effectiveness in deep learning contexts.

Highlights tradeoffs between computational complexity and sample quality.

Abstract

The Fourier Basis Density Model (FBM) was recently introduced as a flexible probability model for band-limited distributions, i.e. ones which are smooth in the sense of having a characteristic function with limited support around the origin. Its density and cumulative distribution functions can be efficiently evaluated and trained with stochastic optimization methods, which makes the model suitable for deep learning applications. However, the model lacked support for sampling. Here, we introduce a method inspired by discretization--interpolation methods common in Digital Signal Processing, which directly take advantage of the band-limited property. We review mathematical properties of the FBM, and prove quality bounds of the sampled distribution in terms of the total variation (TV) and Wasserstein--1 divergences from the model. These bounds can be used to inform the choice of hyperparameters to reach any desired sample quality. We discuss these results in comparison to a variety of other sampling techniques, highlighting tradeoffs between computational complexity and sampling quality.