Importance Weighting and Variational Inference

TL;DR

This paper clarifies the theoretical foundations of importance weighted variational inference (IWVI), demonstrating its connection to augmented variational inference, and explores elliptical distributions to enhance inference accuracy and convergence.

Contribution

It establishes the theoretical link between IWVI and augmented variational inference and investigates elliptical distributions for improved low- and high-dimensional inference.

Findings

IWVI is an instance of augmented variational inference.

Experiments confirm IWVI's practicality for probabilistic inference.

Elliptical distributions improve accuracy and convergence in different dimensions.

Abstract

Recent work used importance sampling ideas for better variational bounds on likelihoods. We clarify the applicability of these ideas to pure probabilistic inference, by showing the resulting Importance Weighted Variational Inference (IWVI) technique is an instance of augmented variational inference, thus identifying the looseness in previous work. Experiments confirm IWVI's practicality for probabilistic inference. As a second contribution, we investigate inference with elliptical distributions, which improves accuracy in low dimensions, and convergence in high dimensions.