Greedy Poisson Rejection Sampling

TL;DR

This paper introduces GPRS, an efficient algorithm for one-shot channel simulation in one-dimensional unimodal density ratio cases, outperforming existing methods in speed and applicability.

Contribution

The paper presents the first optimal runtime algorithm for one-shot channel simulation in unimodal one-dimensional problems using a novel greedy Poisson rejection sampling approach.

Findings

GPRS outperforms A* coding in empirical tests.

The algorithm has proven correctness and optimal time complexity.

The method is applicable to neural data compression and differential privacy.

Abstract

One-shot channel simulation is a fundamental data compression problem concerned with encoding a single sample from a target distribution $Q$ using a coding distribution $P$ using as few bits as possible on average. Algorithms that solve this problem find applications in neural data compression and differential privacy and can serve as a more efficient alternative to quantization-based methods. Sadly, existing solutions are too slow or have limited applicability, preventing widespread adoption. In this paper, we conclusively solve one-shot channel simulation for one-dimensional problems where the target-proposal density ratio is unimodal by describing an algorithm with optimal runtime. We achieve this by constructing a rejection sampling procedure equivalent to greedily searching over the points of a Poisson process. Hence, we call our algorithm greedy Poisson rejection sampling (GPRS) and analyze the correctness and time complexity of several of its variants. Finally, we empirically verify our theorems, demonstrating that GPRS significantly outperforms the current state-of-the-art method, A* coding. Our code is available at https://github.com/gergely-flamich/greedy-poisson-rejection-sampling.