Three Variations on Variational Autoencoders

TL;DR

This paper introduces three novel variations of variational autoencoders, including an Evidence Upper Bound, to improve inference and convergence analysis in generative models.

Contribution

The paper proposes three variations of VAEs with additional encoder/decoder pairs and a fixed encoder, enhancing inference and convergence diagnostics.

Findings

One variation introduces an Evidence Upper Bound (EUBO).

EUBO can be used alongside ELBO to assess convergence.

Comparative analysis shows improved inference capabilities.

Abstract

Variational autoencoders (VAEs) are one class of generative probabilistic latent-variable models designed for inference based on known data. We develop three variations on VAEs by introducing a second parameterized encoder/decoder pair and, for one variation, an additional fixed encoder. The parameters of the encoders/decoders are to be learned with a neural network. The fixed encoder is obtained by probabilistic-PCA. The variations are compared to the Evidence Lower Bound (ELBO) approximation to the original VAE. One variation leads to an Evidence Upper Bound (EUBO) that can be used in conjunction with the original ELBO to interrogate the convergence of the VAE.