A fully Bayesian strategy for high-dimensional hierarchical modeling using massively parallel computing

TL;DR

This paper introduces a fully Bayesian hierarchical modeling approach that leverages massively parallel computing to efficiently perform MCMC, addressing computational challenges in high-dimensional models and enabling scalable inference.

Contribution

The paper presents a novel parallel MCMC algorithm for hierarchical models that reduces computational bottlenecks and is implemented in R packages for practical use.

Findings

Efficient parallelization of MCMC steps for hierarchical models.

Successful application to sequencing data analysis.

Open-source R packages released for implementation.

Abstract

Markov chain Monte Carlo (MCMC) is the predominant tool used in Bayesian parameter estimation for hierarchical models. When the model expands due to an increasing number of hierarchical levels, number of groups at a particular level, or number of observations in each group, a fully Bayesian analysis via MCMC can easily become computationally demanding, even intractable. We illustrate how the steps in an MCMC for hierarchical models are predominantly one of two types: conditionally independent draws or low-dimensional draws based on summary statistics of parameters at higher levels of the hierarchy. Parallel computing can increase efficiency by performing embarrassingly parallel computations for conditionally independent draws and calculating the summary statistics using parallel reductions. During the MCMC algorithm, we record running means and means of squared parameter values to allow convergence diagnosis and posterior inference while avoiding the costly memory transfer bottleneck. We demonstrate the effectiveness of the algorithm on a model motivated by next generation sequencing data, and we release our implementation in R packages fbseq and fbseqCUDA.