Variational Inference via Transformations on Distributions

TL;DR

This paper reviews and implements transformation-based variational inference methods, demonstrating their ability to develop complex posterior approximations and improve learning in Variational Autoencoders on MNIST.

Contribution

It introduces a transformation-based approach to enhance the richness of variational posterior approximations, showing its effectiveness in deep learning models.

Findings

Transformation methods produce richer posterior approximations.

Improved posterior learning in Variational Autoencoders.

Effective on MNIST dataset.

Abstract

Variational inference methods often focus on the problem of efficient model optimization, with little emphasis on the choice of the approximating posterior. In this paper, we review and implement the various methods that enable us to develop a rich family of approximating posteriors. We show that one particular method employing transformations on distributions results in developing very rich and complex posterior approximation. We analyze its performance on the MNIST dataset by implementing with a Variational Autoencoder and demonstrate its effectiveness in learning better posterior distributions.