Probabilistic Programming with CuPPL

TL;DR

This paper introduces CuPPL, a probabilistic programming language and toolchain that generates highly efficient code for CPUs and GPUs, enabling faster inference and broader expressivity in probabilistic models.

Contribution

It presents a novel functional-style probabilistic programming language with a LLVM-based toolchain that supports hardware acceleration on CPUs and GPUs, improving performance.

Findings

Significant speed-ups on CPU workloads compared to state-of-the-art approaches.

Efficient code generation for CUDA GPUs.

Supports highly expressive probabilistic models.

Abstract

Probabilistic Programming Languages (PPLs) are a powerful tool in machine learning, allowing highly expressive generative models to be expressed succinctly. They couple complex inference algorithms, implemented by the language, with an expressive modelling language that allows a user to implement any computable function as the generative model. Such languages are usually implemented on top of existing high level programming languages and do not make use of hardware accelerators. PPLs that do make use of accelerators exist, but restrict the expressivity of the language in order to do so. In this paper, we present a language and toolchain that generates highly efficient code for both CPUs and GPUs. The language is functional in style, and the tool chain is built on top of LLVM. Our implementation uses de-limited continuations on CPU to perform inference, and custom CUDA codes on GPU. We obtain significant speed ups across a suite of PPL workloads, compared to other state of the art approaches on CPU. Furthermore, our compiler can also generate efficient code that runs on CUDA GPUs.