IID Sampling from Intractable Distributions

TL;DR

This paper introduces a new method for generating independent and identically distributed samples from any target distribution in Euclidean space, regardless of support compactness, using an infinite mixture of ellipsoids and perfect samplers, validated across various distributions and dimensions.

Contribution

The paper presents a practically applicable, parallelizable perfect sampling method for any target distribution in Euclidean space, extending previous work by removing support restrictions.

Findings

Successfully generated iid samples from standard and complex distributions in various dimensions.

Achieved high accuracy with reasonable computation time, often under a minute.

Demonstrated applicability to real-world posterior distributions in Bayesian analysis.

Abstract

We propose a novel methodology for drawing iid realizations from any target distribution on the Euclidean space with arbitrary dimension. No assumption of compact support is necessary for the validity of our theory and method. Our idea is to construct an appropriate infinite sequence of concentric closed ellipsoids, represent the target distribution as an infinite mixture on the central ellipsoid and the ellipsoidal annuli, and to construct efficient perfect samplers for the mixture components. In contrast with most of the existing works on perfect sampling, ours is not only a theoretically valid method, it is practically applicable to all target distributions on any dimensional Euclidean space and very much amenable to parallel computation. We validate the practicality and usefulness of our methodology by generating 10000 iid realizations from the standard distributions such as normal, Student's t with 5 degrees of freedom and Cauchy, for dimensions d = 1, 5, 10, 50, 100, as well as from a 50-dimensional mixture normal distribution. The implementation time in all the cases are very reasonable, and often less than a minute in our parallel implementation. The results turned out to be highly accurate. We also apply our method to draw 10000 iid realizations from the posterior distributions associated with the well-known Challenger data, a Salmonella data and the 160-dimensional challenging spatial example of the radionuclide count data on Rongelap Island. Again, we are able to obtain quite encouraging results with very reasonable computing time.