An Easy Rejection Sampling Baseline via Gradient Refined Proposals

TL;DR

This paper introduces a simple, gradient-based rejection sampling method that improves acceptance rates and is easy to implement, requiring only a differentiable target density without complex derivations.

Contribution

The authors propose a new rejection sampling baseline that refines proposals using gradient information, simplifying implementation and improving efficiency without additional assumptions.

Findings

Achieves up to 7.3x higher acceptance rates on benchmarks.

Requires only a differentiable target density, no complex derivations.

Passes distributional checks with high probability.

Abstract

Rejection sampling is a common tool for low dimensional problems ($d \leq 2$), often touted as an "easy" way to obtain valid samples from a distribution $f(\cdot)$ of interest. In practice it is non-trivial to apply, often requiring considerable mathematical effort to devise a good proposal distribution $g(\cdot)$ and select a supremum $C$. More advanced samplers require additional mathematical derivations, limitations on $f(\cdot)$, or even cross-validation, making them difficult to apply. We devise a new approximate baseline approach to rejection sampling that works with less information, requiring only a differentiable $f(\cdot)$ be specified, making it easier to use. We propose a new approach to rejection sampling by refining a parameterized proposal distribution with a loss derived from the acceptance threshold. In this manner we obtain comparable or better acceptance rates on current benchmarks by up to $7.3\times$, while requiring no extra assumptions or any derivations to use: only a differentiable $f(\cdot)$ is required. While approximate, the results are correct with high probability, and in all tests pass a distributional check. This makes our approach easy to use, reproduce, and efficacious.