Variationally Inferred Sampling Through a Refined Bound for Probabilistic Programs

TL;DR

This paper introduces a novel variational inference framework that embeds a sampler within a variational posterior, automatically tuning sampler parameters to improve efficiency in probabilistic programming.

Contribution

It proposes the refined variational approximation that simplifies implementation and enhances sampler tuning via automatic differentiation, with new strategies for ELBO approximation.

Findings

Efficient performance in high-dimensional influence diagram inference

Improved density estimation with unconditional VAE

Effective state-space model inference for time-series data

Abstract

A framework to boost the efficiency of Bayesian inference in probabilistic programs is introduced by embedding a sampler inside a variational posterior approximation. We call it the refined variational approximation. Its strength lies both in ease of implementation and automatically tuning of the sampler parameters to speed up mixing time using automatic differentiation. Several strategies to approximate \emph{evidence lower bound} (ELBO) computation are introduced. Experimental evidence of its efficient performance is shown solving an influence diagram in a high-dimensional space using a conditional variational autoencoder (cVAE) as a deep Bayes classifier; an unconditional VAE on density estimation tasks; and state-space models for time-series data.