Affine Monads and Lazy Structures for Bayesian Programming

TL;DR

This paper introduces a novel semantic framework using affine and non-affine monads to model lazy infinite-dimensional Bayesian methods, supported by new inference algorithms and a Haskell library implementation.

Contribution

It develops a semantic model with two monads for lazy Bayesian programming, enabling new inference methods and a practical Haskell library.

Findings

Supports infinite-dimensional Bayesian models like Gaussian and Dirichlet processes.

Provides correct, inference algorithms adapted for laziness.

Implemented as the LazyPPL Haskell library.

Abstract

We show that streams and lazy data structures are a natural idiom for programming with infinite-dimensional Bayesian methods such as Poisson processes, Gaussian processes, jump processes, Dirichlet processes, and Beta processes. The crucial semantic idea, inspired by developments in synthetic probability theory, is to work with two separate monads: an affine monad of probability, which supports laziness, and a commutative, non-affine monad of measures, which does not. (Affine means that $T(1)\cong 1$.) We show that the separation is important from a decidability perspective, and that the recent model of quasi-Borel spaces supports these two monads. To perform Bayesian inference with these examples, we introduce new inference methods that are specially adapted to laziness; they are proven correct by reference to the Metropolis-Hastings-Green method. Our theoretical development is implemented as a Haskell library, LazyPPL.