Conditional Sampling of Variational Autoencoders via Iterated Approximate Ancestral Sampling

TL;DR

This paper addresses the challenge of conditional sampling in variational autoencoders, proposing novel methods to improve sampling performance and overcome issues caused by structured latent spaces.

Contribution

It introduces two original methods that mitigate pitfalls of Metropolis-within-Gibbs sampling in VAEs, enhancing sampling accuracy and efficiency.

Findings

Proposed methods outperform standard MWG in sampling tasks.

Structured latent spaces can cause MWG to get stuck, which our methods address.

Improved sampling performance demonstrated on multiple tasks.

Abstract

Conditional sampling of variational autoencoders (VAEs) is needed in various applications, such as missing data imputation, but is computationally intractable. A principled choice for asymptotically exact conditional sampling is Metropolis-within-Gibbs (MWG). However, we observe that the tendency of VAEs to learn a structured latent space, a commonly desired property, can cause the MWG sampler to get "stuck" far from the target distribution. This paper mitigates the limitations of MWG: we systematically outline the pitfalls in the context of VAEs, propose two original methods that address these pitfalls, and demonstrate an improved performance of the proposed methods on a set of sampling tasks.