Instance Dependent Testing of Samplers using Interval Conditioning

TL;DR

This paper introduces a novel instance-dependent testing method for samplers over infinite domains, utilizing interval conditioning to achieve significant efficiency improvements in verifying sampler correctness.

Contribution

It presents the first instance-dependent sampler tester over natural numbers, leveraging a new distance estimation algorithm and a novel connection to continuous distribution mass estimation.

Findings

Achieves up to 1000x speedup over existing testers

Supports verification over infinite domains like natural numbers

Introduces a new interval conditioning framework for distribution testing

Abstract

Sampling algorithms play a pivotal role in probabilistic AI. However, verifying if a sampler program indeed samples from the claimed distribution is a notoriously hard problem. Provably correct testers like Barbarik, Teq, Flash, CubeProbe for testing of different kinds of samplers were proposed only in the last few years. All these testers focus on the worst-case efficiency, and do not support verification of samplers over infinite domains, a case occurring frequently in Astronomy, Finance, Network Security, etc. In this work, we design the first tester of samplers with instance-dependent efficiency, allowing us to test samplers over natural numbers. Our tests are developed via a novel distance estimation algorithm between an unknown and a known probability distribution using an interval conditioning framework. The core technical contribution is a new connection with probability mass estimation of a continuous distribution. The practical gains are also substantial: our experiments establish up to 1000x speedup over state-of-the-art testers.