How To Train Your Program: a Probabilistic Programming Pattern for Bayesian Learning From Data

TL;DR

This paper introduces a probabilistic programming pattern called 'stump and fungus' for Bayesian learning, enabling efficient inference on new data by reusing previous posterior distributions, thus reducing computational costs.

Contribution

It presents a novel design pattern for probabilistic programming that facilitates Bayesian updating with lower computational costs by reusing previous inference results.

Findings

Pattern effectively reuses previous inference for new data

Reduces computational cost compared to full hierarchical inference

Validated on multiple case studies

Abstract

We present a Bayesian approach to machine learning with probabilistic programs. In our approach, training on available data is implemented as inference on a hierarchical model. The posterior distribution of model parameters is then used to \textit{stochastically condition} a complementary model, such that inference on new data yields the same posterior distribution of latent parameters corresponding to the new data as inference on a hierachical model on the combination of both previously available and new data, at a lower computation cost. We frame the approach as a design pattern of probabilistic programming referred to herein as `stump and fungus', and evaluate realization of the pattern on case studies.