Isolating Sources of Disentanglement in Variational Autoencoders

TL;DR

This paper introduces a new variational autoencoder variant, the $eta$-TCVAE, which explicitly minimizes total correlation to improve disentanglement of latent representations, along with a classifier-free disentanglement measure called MIG.

Contribution

It proposes the $eta$-TCVAE that isolates total correlation in the objective and introduces MIG as a new disentanglement metric, advancing understanding and measurement of disentanglement.

Findings

Total correlation correlates strongly with disentanglement.

The $eta$-TCVAE outperforms previous models in disentanglement metrics.

MIG provides a classifier-free measure of disentanglement.

Abstract

We decompose the evidence lower bound to show the existence of a term measuring the total correlation between latent variables. We use this to motivate our $\beta$-TCVAE (Total Correlation Variational Autoencoder), a refinement of the state-of-the-art $\beta$-VAE objective for learning disentangled representations, requiring no additional hyperparameters during training. We further propose a principled classifier-free measure of disentanglement called the mutual information gap (MIG). We perform extensive quantitative and qualitative experiments, in both restricted and non-restricted settings, and show a strong relation between total correlation and disentanglement, when the latent variables model is trained using our framework.