On the utility of graphics cards to perform massively parallel simulation of advanced Monte Carlo methods

TL;DR

This paper demonstrates that graphics cards can massively accelerate advanced Monte Carlo simulations, achieving 35 to 500 times speedup, thus enabling complex statistical modeling in accessible and cost-effective ways.

Contribution

It provides empirical evidence of significant speedups in Monte Carlo methods using GPUs, highlighting their potential for scalable and efficient statistical computation.

Findings

Speedups of 35 to 500 fold over single-threaded code.

GPUs are cost-effective, accessible, and easy to maintain for parallel simulations.

Potential to expand statistical modeling into complex, data-rich domains.

Abstract

We present a case-study on the utility of graphics cards to perform massively parallel simulation of advanced Monte Carlo methods. Graphics cards, containing multiple Graphics Processing Units (GPUs), are self-contained parallel computational devices that can be housed in conventional desktop and laptop computers. For certain classes of Monte Carlo algorithms they offer massively parallel simulation, with the added advantage over conventional distributed multi-core processors that they are cheap, easily accessible, easy to maintain, easy to code, dedicated local devices with low power consumption. On a canonical set of stochastic simulation examples including population-based Markov chain Monte Carlo methods and Sequential Monte Carlo methods, we find speedups from 35 to 500 fold over conventional single-threaded computer code. Our findings suggest that GPUs have the potential to facilitate the growth of statistical modelling into complex data rich domains through the availability of cheap and accessible many-core computation. We believe the speedup we observe should motivate wider use of parallelizable simulation methods and greater methodological attention to their design.