Robust Disentanglement of a Few Factors at a Time

TL;DR

This paper presents a recursive training method for variational autoencoders that improves unsupervised disentanglement of data factors by iteratively removing learned factors and retraining, leading to state-of-the-art results.

Contribution

It introduces the recursive rPU-VAE approach and the use of Unsupervised Disentanglement Ranking to enhance and stabilize unsupervised disentanglement.

Findings

Significant improvement in disentanglement performance.

Enhanced robustness across datasets.

Effective iterative removal of learned factors.

Abstract

Disentanglement is at the forefront of unsupervised learning, as disentangled representations of data improve generalization, interpretability, and performance in downstream tasks. Current unsupervised approaches remain inapplicable for real-world datasets since they are highly variable in their performance and fail to reach levels of disentanglement of (semi-)supervised approaches. We introduce population-based training (PBT) for improving consistency in training variational autoencoders (VAEs) and demonstrate the validity of this approach in a supervised setting (PBT-VAE). We then use Unsupervised Disentanglement Ranking (UDR) as an unsupervised heuristic to score models in our PBT-VAE training and show how models trained this way tend to consistently disentangle only a subset of the generative factors. Building on top of this observation we introduce the recursive rPU-VAE approach. We train the model until convergence, remove the learned factors from the dataset and reiterate. In doing so, we can label subsets of the dataset with the learned factors and consecutively use these labels to train one model that fully disentangles the whole dataset. With this approach, we show striking improvement in state-of-the-art unsupervised disentanglement performance and robustness across multiple datasets and metrics.