On Causally Disentangled Representations

TL;DR

This paper introduces a causal perspective on disentangled representations, proposing new metrics and datasets to evaluate how well models capture the underlying causal generative process, and empirically assesses state-of-the-art models.

Contribution

It presents a causal framework for disentanglement, new evaluation metrics, a dataset, and an empirical study of existing models from a causal perspective.

Findings

Metrics effectively capture causal disentanglement

State-of-the-art models vary in causal fidelity

Proposed dataset enables causal evaluation

Abstract

Representation learners that disentangle factors of variation have already proven to be important in addressing various real world concerns such as fairness and interpretability. Initially consisting of unsupervised models with independence assumptions, more recently, weak supervision and correlated features have been explored, but without a causal view of the generative process. In contrast, we work under the regime of a causal generative process where generative factors are either independent or can be potentially confounded by a set of observed or unobserved confounders. We present an analysis of disentangled representations through the notion of disentangled causal process. We motivate the need for new metrics and datasets to study causal disentanglement and propose two evaluation metrics and a dataset. We show that our metrics capture the desiderata of disentangled causal process. Finally, we perform an empirical study on state of the art disentangled representation learners using our metrics and dataset to evaluate them from causal perspective.