Weakly Supervised Disentanglement with Guarantees

TL;DR

This paper introduces a theoretical framework to analyze when weak supervision guarantees disentangled representations in machine learning, supported by empirical validation of various weak supervision methods.

Contribution

It provides the first formal analysis of the conditions under which weak supervision guarantees disentanglement in learned representations.

Findings

Theoretical framework predicts when weak supervision guarantees disentanglement.

Empirical results verify the framework's predictive accuracy.

Analysis of methods like restricted labeling, match-pairing, and rank-pairing.

Abstract

Learning disentangled representations that correspond to factors of variation in real-world data is critical to interpretable and human-controllable machine learning. Recently, concerns about the viability of learning disentangled representations in a purely unsupervised manner has spurred a shift toward the incorporation of weak supervision. However, there is currently no formalism that identifies when and how weak supervision will guarantee disentanglement. To address this issue, we provide a theoretical framework to assist in analyzing the disentanglement guarantees (or lack thereof) conferred by weak supervision when coupled with learning algorithms based on distribution matching. We empirically verify the guarantees and limitations of several weak supervision methods (restricted labeling, match-pairing, and rank-pairing), demonstrating the predictive power and usefulness of our theoretical framework.