Functional probabilistic programming for scalable Bayesian modelling

TL;DR

This paper presents a functional programming approach to probabilistic programming, enabling scalable Bayesian inference with flexible inference algorithms like Hamiltonian Monte Carlo, demonstrated in Scala.

Contribution

It introduces a functional programming framework for probabilistic programming that decouples model specification from inference algorithms, enhancing scalability and flexibility.

Findings

Uses automatic differentiation for efficient gradient computation.

Applies the framework to scalable Bayesian models in Scala.

Enables flexible inference with generic algorithms like HMC.

Abstract

Bayesian inference involves the specification of a statistical model by a statistician or practitioner, with careful thought about what each parameter represents. This results in particularly interpretable models which can be used to explain relationships present in the observed data. Bayesian models are useful when an experiment has only a small number of observations and in applications where transparency of data driven decisions is important. Traditionally, parameter inference in Bayesian statistics has involved constructing bespoke MCMC (Markov chain Monte Carlo) schemes for each newly proposed statistical model. This results in plausible models not being considered since efficient inference schemes are challenging to develop or implement. Probabilistic programming aims to reduce the barrier to performing Bayesian inference by developing a domain specific language (DSL) for model specification which is decoupled from the parameter inference algorithms. This paper introduces functional programming principles which can be used to develop an embedded probabilistic programming language. Model inference can be carried out using any generic inference algorithm. In this paper Hamiltonian Monte Carlo (HMC) is used, an efficient MCMC method requiring the gradient of the un-normalised log-posterior, calculated using automatic differentiation. The concepts are illustrated using the Scala programming language.