Pigeons.jl: Distributed Sampling From Intractable Distributions

TL;DR

Pigeons.jl is a software package enabling distributed sampling from complex probability distributions, ensuring reproducibility across different computational setups, and supporting scalable Bayesian and statistical mechanics applications.

Contribution

It introduces Pigeons.jl, a novel software tool that guarantees strong parallelism invariance for distributed sampling from intractable distributions.

Findings

Supports sampling from multimodal and high-dimensional distributions

Ensures reproducibility regardless of computational parallelism

Provides scalable APIs for distributed computation

Abstract

We introduce a software package, Pigeons.jl, that provides a way to leverage distributed computation to obtain samples from complicated probability distributions, such as multimodal posteriors arising in Bayesian inference and high-dimensional distributions in statistical mechanics. Pigeons.jl provides simple APIs to perform such computations single-threaded, multi-threaded, and/or distributed over thousands of MPI-communicating machines. In addition, Pigeons.jl guarantees a property that we call strong parallelism invariance: the output for a given seed is identical irrespective of the number of threads and processes, which is crucial for scientific reproducibility and software validation. We describe the key features of Pigeons.jl and the approach taken to implement a distributed and randomized algorithm that satisfies strong parallelism invariance.