Weakly-Supervised Disentanglement Without Compromises

TL;DR

This paper demonstrates that weakly-supervised learning from pairs of images sharing some factors can effectively produce disentangled representations without detailed annotations, benefiting various downstream tasks.

Contribution

It introduces theoretical insights and practical algorithms for learning disentangled representations from image pairs with minimal supervision, without needing explicit factor annotations.

Findings

Disentangled representations can be learned from pairs of images sharing factors.

The approach performs well on multiple benchmark datasets.

Learned representations improve generalization, fairness, and reasoning tasks.

Abstract

Intelligent agents should be able to learn useful representations by observing changes in their environment. We model such observations as pairs of non-i.i.d. images sharing at least one of the underlying factors of variation. First, we theoretically show that only knowing how many factors have changed, but not which ones, is sufficient to learn disentangled representations. Second, we provide practical algorithms that learn disentangled representations from pairs of images without requiring annotation of groups, individual factors, or the number of factors that have changed. Third, we perform a large-scale empirical study and show that such pairs of observations are sufficient to reliably learn disentangled representations on several benchmark data sets. Finally, we evaluate our learned representations and find that they are simultaneously useful on a diverse suite of tasks, including generalization under covariate shifts, fairness, and abstract reasoning. Overall, our results demonstrate that weak supervision enables learning of useful disentangled representations in realistic scenarios.