Statically Bounded-Memory Delayed Sampling for Probabilistic Streams

TL;DR

This paper establishes conditions and a static analysis method to ensure bounded memory usage for delayed sampling in probabilistic programming streams, improving memory guarantees in probabilistic inference.

Contribution

It introduces specific dataflow properties that guarantee bounded memory in delayed sampling and develops a static analysis to verify these properties in probabilistic programs.

Findings

Conditions for bounded memory execution are identified.

A static analysis method is proposed to verify these conditions.

The approach ensures bounded memory in probabilistic streaming applications.

Abstract

Probabilistic programming languages aid developers performing Bayesian inference. These languages provide programming constructs and tools for probabilistic modeling and automated inference. Prior work introduced a probabilistic programming language, ProbZelus, to extend probabilistic programming functionality to unbounded streams of data. This work demonstrated that the delayed sampling inference algorithm could be extended to work in a streaming context. ProbZelus showed that while delayed sampling could be effectively deployed on some programs, depending on the probabilistic model under consideration, delayed sampling is not guaranteed to use a bounded amount of memory over the course of the execution of the program. In this paper, we present conditions on a probabilistic program's execution under which delayed sampling will execute in bounded memory. The two conditions are dataflow properties of the core operations of delayed sampling: the $m$-consumed property and the unseparated paths property. A program executes in bounded memory under delayed sampling if, and only if, it satisfies the $m$-consumed and unseparated paths properties. We propose a static analysis that abstracts over these properties to soundly ensure that any program that passes the analysis satisfies these properties, and thus executes in bounded memory under delayed sampling.