Deriving Probability Density Functions from Probabilistic Functional Programs

TL;DR

This paper introduces a density compiler for probabilistic functional programs that simplifies deriving probability density functions, enabling efficient inference in scientific applications.

Contribution

It presents a sound density compiler for a probabilistic language with failure and mixed distributions, advancing the automation of density derivation.

Findings

Successfully applied to scientific inference problems

Reduces development effort for probabilistic modeling

Enables integration with MCMC methods

Abstract

The probability density function of a probability distribution is a fundamental concept in probability theory and a key ingredient in various widely used machine learning methods. However, the necessary framework for compiling probabilistic functional programs to density functions has only recently been developed. In this work, we present a density compiler for a probabilistic language with failure and both discrete and continuous distributions, and provide a proof of its soundness. The compiler greatly reduces the development effort of domain experts, which we demonstrate by solving inference problems from various scientific applications, such as modelling the global carbon cycle, using a standard Markov chain Monte Carlo framework.