Scalable Rejection Sampling for Bayesian Hierarchical Models

TL;DR

This paper introduces a scalable rejection sampling method for Bayesian hierarchical models that produces independent samples without MCMC, enabling efficient analysis of large datasets with conditionally independent units.

Contribution

The authors develop a novel rejection sampling algorithm that is scalable, parallelizable, and avoids MCMC limitations for Bayesian hierarchical models.

Findings

Samples are independent and can be collected in parallel.

The method is scalable for large datasets with conditionally independent units.

It can be used to compute marginal likelihoods efficiently.

Abstract

Bayesian hierarchical modeling is a popular approach to capturing unobserved heterogeneity across individual units. However, standard estimation methods such as Markov chain Monte Carlo (MCMC) can be impracticable for modeling outcomes from a large number of units. We develop a new method to sample from posterior distributions of Bayesian models, without using MCMC. Samples are independent, so they can be collected in parallel, and we do not need to be concerned with issues like chain convergence and autocorrelation. The algorithm is scalable under the weak assumption that individual units are conditionally independent, making it applicable for large datasets. It can also be used to compute marginal likelihoods.