Compiling Stan to Generative Probabilistic Languages and Extension to Deep Probabilistic Programming

TL;DR

This paper develops a compilation scheme to translate Stan models into generative probabilistic languages, enabling new features like deep probabilistic models and variational inference guides while improving execution speed.

Contribution

It introduces a formal compilation method from Stan to generative languages and extends Stan with deep probabilistic modeling capabilities.

Findings

NumPyro backend is 2.3x faster than Stan on average.

The compilation scheme is proven correct.

Stan can now support deep probabilistic models and variational guides.

Abstract

Stan is a probabilistic programming language that is popular in the statistics community, with a high-level syntax for expressing probabilistic models. Stan differs by nature from generative probabilistic programming languages like Church, Anglican, or Pyro. This paper presents a comprehensive compilation scheme to compile any Stan model to a generative language and proves its correctness. We use our compilation scheme to build two new backends for the Stanc3 compiler targeting Pyro and NumPyro. Experimental results show that the NumPyro backend yields a 2.3x speedup compared to Stan in geometric mean over 26 benchmarks. Building on Pyro we extend Stan with support for explicit variational inference guides and deep probabilistic models. That way, users familiar with Stan get access to new features without having to learn a fundamentally new language.