C-Disentanglement: Discovering Causally-Independent Generative Factors under an Inductive Bias of Confounder

TL;DR

This paper introduces C-Disentanglement, a novel framework that leverages domain knowledge of confounders to identify causally disentangled generative factors, improving robustness and generalization in representation learning.

Contribution

It is the first to explicitly incorporate confounder inductive bias via labels, enabling identification of causally disentangled factors under confounded data.

Findings

Outperforms state-of-the-art methods in causally disentangled feature discovery

Demonstrates robustness under domain shifts in real-world datasets

Achieves competitive results in downstream tasks

Abstract

Representation learning assumes that real-world data is generated by a few semantically meaningful generative factors (i.e., sources of variation) and aims to discover them in the latent space. These factors are expected to be causally disentangled, meaning that distinct factors are encoded into separate latent variables, and changes in one factor will not affect the values of the others. Compared to statistical independence, causal disentanglement allows more controllable data generation, improved robustness, and better generalization. However, most existing work assumes unconfoundedness in the discovery process, that there are no common causes to the generative factors and thus obtain only statistical independence. In this paper, we recognize the importance of modeling confounders in discovering causal generative factors. Unfortunately, such factors are not identifiable without proper inductive bias. We fill the gap by introducing a framework entitled Confounded-Disentanglement (C-Disentanglement), the first framework that explicitly introduces the inductive bias of confounder via labels from domain expertise. In addition, we accordingly propose an approach to sufficiently identify the causally disentangled factors under any inductive bias of the confounder. We conduct extensive experiments on both synthetic and real-world datasets. Our method demonstrates competitive results compared to various SOTA baselines in obtaining causally disentangled features and downstream tasks under domain shifts.