InfoVAE: Information Maximizing Variational Autoencoders

TL;DR

InfoVAE introduces a new training objective for variational autoencoders that enhances inference quality and latent variable utilization, outperforming existing methods across various metrics.

Contribution

The paper proposes InfoVAE, a novel class of objectives that address inference inaccuracies and latent variable neglect in VAEs, improving their performance.

Findings

Significant improvement in variational posterior quality.

Effective use of latent features regardless of decoder flexibility.

Outperforms competing approaches on multiple metrics.

Abstract

A key advance in learning generative models is the use of amortized inference distributions that are jointly trained with the models. We find that existing training objectives for variational autoencoders can lead to inaccurate amortized inference distributions and, in some cases, improving the objective provably degrades the inference quality. In addition, it has been observed that variational autoencoders tend to ignore the latent variables when combined with a decoding distribution that is too flexible. We again identify the cause in existing training criteria and propose a new class of objectives (InfoVAE) that mitigate these problems. We show that our model can significantly improve the quality of the variational posterior and can make effective use of the latent features regardless of the flexibility of the decoding distribution. Through extensive qualitative and quantitative analyses, we demonstrate that our models outperform competing approaches on multiple performance metrics.