A Closer Look at Disentangling in $\beta$-VAE

TL;DR

This paper investigates the limitations of $eta$-VAE in learning disentangled representations, revealing that an optimal $eta$ exists due to a fundamental incompatibility between independence constraints.

Contribution

It provides analytical and numerical analysis showing the non-monotonic inference performance of $eta$-VAE caused by the conflicting independence conditions.

Findings

$eta$-VAE has a finite optimal $eta$ for disentanglement.

Conditional independence enforced by $eta$-VAE conflicts with latent independence.

Performance of $eta$-VAE is non-monotonic with respect to $eta$.

Abstract

In many data analysis tasks, it is beneficial to learn representations where each dimension is statistically independent and thus disentangled from the others. If data generating factors are also statistically independent, disentangled representations can be formed by Bayesian inference of latent variables. We examine a generalization of the Variational Autoencoder (VAE), $\beta$-VAE, for learning such representations using variational inference. $\beta$-VAE enforces conditional independence of its bottleneck neurons controlled by its hyperparameter $\beta$. This condition is in general not compatible with the statistical independence of latents. By providing analytical and numerical arguments, we show that this incompatibility leads to a non-monotonic inference performance in $\beta$-VAE with a finite optimal $\beta$.