GPU-accelerated Gibbs sampling: a case study of the Horseshoe Probit model

TL;DR

This paper presents a GPU-accelerated, fully data-parallel Gibbs sampling method for exchangeable models with latent variables, demonstrated on a Horseshoe Probit model, achieving scalable performance on large datasets.

Contribution

It introduces a novel GPU-based parallel Gibbs sampling approach for a broad class of models, improving scalability and efficiency in Bayesian inference.

Findings

Scales effectively to thousands of predictors

Handles millions of data points efficiently

Achieves significant speedup over traditional methods

Abstract

Gibbs sampling is a widely used Markov chain Monte Carlo (MCMC) method for numerically approximating integrals of interest in Bayesian statistics and other mathematical sciences. Many implementations of MCMC methods do not extend easily to parallel computing environments, as their inherently sequential nature incurs a large synchronization cost. In the case study illustrated by this paper, we show how to do Gibbs sampling in a fully data-parallel manner on a graphics processing unit, for a large class of exchangeable models that admit latent variable representations. Our approach takes a systems perspective, with emphasis placed on efficient use of compute hardware. We demonstrate our method on a Horseshoe Probit regression model and find that our implementation scales effectively to thousands of predictors and millions of data points simultaneously.