Identifiability Guarantees for Causal Disentanglement from Purely Observational Data

TL;DR

This paper establishes theoretical guarantees for identifying causal factors from purely observational data in nonlinear models with additive Gaussian noise, and proposes a practical algorithm for causal disentanglement.

Contribution

It provides the first precise characterization of what causal factors can be identified without interventions in nonlinear models, and introduces a quadratic programming algorithm for this purpose.

Findings

Causal variables can be identified up to a layer-wise transformation.

Further disentanglement beyond this layer-wise transformation is impossible.

The proposed algorithm successfully derives meaningful causal representations from observational data.

Abstract

Causal disentanglement aims to learn about latent causal factors behind data, holding the promise to augment existing representation learning methods in terms of interpretability and extrapolation. Recent advances establish identifiability results assuming that interventions on (single) latent factors are available; however, it remains debatable whether such assumptions are reasonable due to the inherent nature of intervening on latent variables. Accordingly, we reconsider the fundamentals and ask what can be learned using just observational data. We provide a precise characterization of latent factors that can be identified in nonlinear causal models with additive Gaussian noise and linear mixing, without any interventions or graphical restrictions. In particular, we show that the causal variables can be identified up to a layer-wise transformation and that further disentanglement is not possible. We transform these theoretical results into a practical algorithm consisting of solving a quadratic program over the score estimation of the observed data. We provide simulation results to support our theoretical guarantees and demonstrate that our algorithm can derive meaningful causal representations from purely observational data.