Verifying Handcoded Probabilistic Inference Procedures

TL;DR

This paper introduces Shuffle, a programming language that helps developers create correct and efficient probabilistic inference algorithms by enforcing probability rules and dependencies, demonstrated on standard models.

Contribution

Shuffle is a novel language that enforces correctness and optimizes manual probabilistic inference procedures, bridging the gap between manual development and correctness guarantees.

Findings

Shuffle enforces probability rules and dependencies.

It generates optimized inference implementations.

Developers can produce correct, high-performance algorithms.

Abstract

Researchers have recently proposed several systems that ease the process of performing Bayesian probabilistic inference. These include systems for automatic inference algorithm synthesis as well as stronger abstractions for manual algorithm development. However, existing systems whose performance relies on the developer manually constructing a part of the inference algorithm have limited support for reasoning about the correctness of the resulting algorithm. In this paper, we present Shuffle, a programming language for manually developing inference procedures that 1) enforces the basic rules of probability theory, 2) enforces the statistical dependencies of the algorithm's corresponding probabilistic model, and 3) generates an optimized implementation. We have used Shuffle to develop inference algorithms for several standard probabilistic models. Our results demonstrate that Shuffle enables a developer to deliver correct and performant implementations of these algorithms.