Automatic Guide Generation for Stan via NumPyro

TL;DR

This paper presents a compiler that translates Stan models into Pyro, enabling Stan users to access a broader set of variational inference algorithms and compare their performance on a standard benchmark.

Contribution

The authors introduce a compiler from Stan to Pyro, allowing seamless access to Pyro's variational inference algorithms for Stan models.

Findings

Pyro's algorithms provide various trade-offs between complexity and accuracy.

The compiler enables leveraging new inference features for Stan users.

Evaluation on PosteriorDB demonstrates practical benefits of the approach.

Abstract

Stan is a very popular probabilistic language with a state-of-the-art HMC sampler but it only offers a limited choice of algorithms for black-box variational inference. In this paper, we show that using our recently proposed compiler from Stan to Pyro, Stan users can easily try the set of algorithms implemented in Pyro for black-box variational inference. We evaluate our approach on PosteriorDB, a database of Stan models with corresponding data and reference posterior samples. Results show that the eight algorithms available in Pyro offer a range of possible compromises between complexity and accuracy. This paper illustrates that compiling Stan to another probabilistic language can be used to leverage new features for Stan users, and give access to a large set of examples for language developers who implement these new features.