GPU-based Parallel Computation Support for Stan

TL;DR

This paper introduces an OpenCL framework enabling Stan to leverage GPUs for faster computation, including optimized routines for matrix operations and likelihoods, demonstrated through logistic and Gaussian Process regressions.

Contribution

It presents an extensible GPU support framework for Stan, allowing seamless acceleration of Bayesian inference tasks with minimal user effort.

Findings

Significant speedups in logistic regression and Gaussian Process regression.

GPU routines for matrix algebra primitives and likelihoods improve computational efficiency.

Framework is easily extensible for additional routines and models.

Abstract

This paper details an extensible OpenCL framework that allows Stan to utilize heterogeneous compute devices. It includes GPU-optimized routines for the Cholesky decomposition, its derivative, other matrix algebra primitives and some commonly used likelihoods, with more additions planned for the near future. Stan users can now benefit from large speedups offered by GPUs with little effort and without changes to their existing Stan code. We demonstrate the practical utility of our work with two examples - logistic regression and Gaussian Process regression.