Nested Slice Sampling: Vectorized Nested Sampling for GPU-Accelerated Inference

TL;DR

Nested Slice Sampling (NSS) is a GPU-optimized, vectorized nested sampling method that efficiently handles complex, multimodal Bayesian inference problems, providing accurate evidence estimates and robust posterior sampling.

Contribution

The paper introduces NSS, a novel GPU-friendly, vectorized nested sampling algorithm that improves efficiency and robustness for high-dimensional, multimodal Bayesian inference tasks.

Findings

NSS maintains accurate evidence estimates across various challenging targets.

NSS outperforms state-of-the-art methods like tempered SMC on difficult multimodal problems.

The implementation is open-source, promoting adoption and reproducibility.

Abstract

Model comparison and calibrated uncertainty quantification often require integrating over parameters, but scalable inference can be challenging for complex, multimodal targets. Nested Sampling is a robust alternative to standard MCMC, yet its typically sequential structure and hard constraints make efficient accelerator implementations difficult. This paper introduces Nested Slice Sampling (NSS), a GPU-friendly, vectorized formulation of Nested Sampling that uses Hit-and-Run Slice Sampling for constrained updates. A tuning analysis yields a simple near-optimal rule for setting the slice width, improving high-dimensional behavior and making per-step compute more predictable for parallel execution. Experiments on challenging synthetic targets, high dimensional Bayesian inference, and Gaussian process hyperparameter marginalization show that NSS maintains accurate evidence estimates and high-quality posterior samples, and is particularly robust on difficult multimodal problems where current state-of-the-art methods such as tempered SMC baselines can struggle. An open-source implementation is released to facilitate adoption and reproducibility.