Automated Parameter Blocking for Efficient Markov-Chain Monte Carlo Sampling

TL;DR

This paper introduces an automated method to optimize Markov-Chain Monte Carlo (MCMC) sampling by dynamically determining parameter blocks for joint sampling, significantly improving efficiency across various models.

Contribution

It presents the first automated procedure for parameter blocking in MCMC, enhancing sampling efficiency without manual intervention.

Findings

Non-trivial efficiency improvements observed across diverse models

Automated blocking reduces MCMC runtime significantly

Procedure is adaptable to different models and platforms

Abstract

Markov chain Monte Carlo (MCMC) sampling is an important and commonly used tool for the analysis of hierarchical models. Nevertheless, practitioners generally have two options for MCMC: utilize existing software that generates a black-box "one size fits all" algorithm, or the challenging (and time consuming) task of implementing a problem-specific MCMC algorithm. Either choice may result in inefficient sampling, and hence researchers have become accustomed to MCMC runtimes on the order of days (or longer) for large models. We propose an automated procedure to determine an efficient MCMC algorithm for a given model and computing platform. Our procedure dynamically determines blocks of parameters for joint sampling that result in efficient sampling of the entire model. We test this procedure using a diverse suite of example models, and observe non-trivial improvements in MCMC efficiency for many models. Our procedure is the first attempt at such, and may be generalized to a broader space of MCMC algorithms. Our results suggest that substantive improvements in MCMC efficiency may be practically realized using our automated blocking procedure, or variants thereof, which warrants additional study and application.