Automated Variational Inference in Probabilistic Programming

TL;DR

This paper introduces an efficient, restriction-free variational inference algorithm for probabilistic programming that automatically derives and optimizes mean-field programs, improving inference efficiency for complex, intractable distributions.

Contribution

It presents a novel stochastic gradient-based variational inference method applicable to general probabilistic programs, with automatic derivation and optimization of mean-field models.

Findings

Improved inference efficiency over existing algorithms

Applicable to highly structured, intractable distributions

Automatic derivation of mean-field probabilistic programs

Abstract

We present a new algorithm for approximate inference in probabilistic programs, based on a stochastic gradient for variational programs. This method is efficient without restrictions on the probabilistic program; it is particularly practical for distributions which are not analytically tractable, including highly structured distributions that arise in probabilistic programs. We show how to automatically derive mean-field probabilistic programs and optimize them, and demonstrate that our perspective improves inference efficiency over other algorithms.