Multi-Language Probabilistic Programming

TL;DR

This paper introduces MultiPPL, a multi-language probabilistic programming framework that allows seamless interoperability between exact and approximate inference languages, enhancing flexibility and performance in complex probabilistic programs.

Contribution

It presents the first sound foundation for multi-language probabilistic programming, combining two distinct inference strategies within a unified syntax and semantics.

Findings

Enables inference on complex heterogeneous probabilistic programs

Proves soundness of the inference algorithm

Empirically demonstrates flexible exploitation of language strengths

Abstract

There are many different probabilistic programming languages that are specialized to specific kinds of probabilistic programs. From a usability and scalability perspective, this is undesirable: today, probabilistic programmers are forced up-front to decide which language they want to use and cannot mix-and-match different languages for handling heterogeneous programs. To rectify this, we seek a foundation for sound interoperability for probabilistic programming languages: just as today's Python programmers can resort to low-level C programming for performance, we argue that probabilistic programmers should be able to freely mix different languages for meeting the demands of heterogeneous probabilistic programming environments. As a first step towards this goal, we introduce \textsc{MultiPPL}, a probabilistic multi-language that enables programmers to interoperate between two different probabilistic programming languages: one that leverages a high-performance exact discrete inference strategy, and one that uses approximate importance sampling. We give a syntax and semantics for \textsc{MultiPPL}, prove soundness of its inference algorithm, and provide empirical evidence that it enables programmers to perform inference on complex heterogeneous probabilistic programs and flexibly exploits the strengths and weaknesses of two languages simultaneously.%