Some Notes on the Sample Complexity of Approximate Channel Simulation

TL;DR

This paper investigates the sample complexity of approximate channel simulation, showing super-polynomial runtime lower bounds and proposing an A* coding method that achieves low total variation distance with exponential sample complexity under certain conditions.

Contribution

It strengthens existing lower bounds on runtime for approximate schemes and introduces an A* coding approach that leverages Radon-Nikodym derivatives and KL divergence for efficient approximation.

Findings

Super-polynomial runtime lower bound for certain distribution pairs.

A* coding achieves TV distance ≤ ε with exponential sample complexity.

Utilizes Radon-Nikodym derivatives and KL divergence for improved efficiency.

Abstract

Channel simulation algorithms can efficiently encode random samples from a prescribed target distribution $Q$ and find applications in machine learning-based lossy data compression. However, algorithms that encode exact samples usually have random runtime, limiting their applicability when a consistent encoding time is desirable. Thus, this paper considers approximate schemes with a fixed runtime instead. First, we strengthen a result of Agustsson and Theis and show that there is a class of pairs of target distribution $Q$ and coding distribution $P$, for which the runtime of any approximate scheme scales at least super-polynomially in $D_\infty[Q \Vert P]$. We then show, by contrast, that if we have access to an unnormalised Radon-Nikodym derivative $r \propto dQ/dP$ and knowledge of $D_{KL}[Q \Vert P]$, we can exploit global-bound, depth-limited A* coding to ensure $\mathrm{TV}[Q \Vert P] \leq \epsilon$ and maintain optimal coding performance with a sample complexity of only $\exp_2\big((D_{KL}[Q \Vert P] + o(1)) \big/ \epsilon\big)$.