A Dynamic Programming Algorithm for Inference in Recursive Probabilistic Programs

TL;DR

This paper introduces a dynamic programming algorithm that efficiently computes marginal distributions in recursive probabilistic programs by constructing a dependency graph and solving fixed-point equations.

Contribution

It presents a novel method to handle recursion in probabilistic programming by building a dependency graph and solving marginal distributions through fixed-point iteration.

Findings

Algorithm effectively computes marginals in recursive probabilistic programs.

The approach handles cyclic dependencies via fixed-point iteration.

Demonstrated on examples from teaching, cognitive science, and game theory.

Abstract

We describe a dynamic programming algorithm for computing the marginal distribution of discrete probabilistic programs. This algorithm takes a functional interpreter for an arbitrary probabilistic programming language and turns it into an efficient marginalizer. Because direct caching of sub-distributions is impossible in the presence of recursion, we build a graph of dependencies between sub-distributions. This factored sum-product network makes (potentially cyclic) dependencies between subproblems explicit, and corresponds to a system of equations for the marginal distribution. We solve these equations by fixed-point iteration in topological order. We illustrate this algorithm on examples used in teaching probabilistic models, computational cognitive science research, and game theory.