Discrete Causal Representation Learning

TL;DR

This paper introduces Discrete Causal Representation Learning (DCRL), a framework for uncovering causal structures among discrete latent variables from complex observations, with theoretical guarantees and practical algorithms.

Contribution

DCRL models causal graphs among discrete latents with a flexible, interpretable generative approach, and provides provable identifiability and a consistent discovery pipeline.

Findings

DCRL accurately recovers sparse causal structures in experiments.

The method handles mixed-type observations effectively.

Theoretical guarantees ensure reliable latent causal discovery.

Abstract

Causal representation learning seeks to uncover causal relationships among high-level latent variables from low-level, entangled, and noisy observations. Existing approaches often either rely on deep neural networks, which lack interpretability and formal guarantees, or impose restrictive assumptions like linearity, continuous-only observations, and strong structural priors. These limitations particularly challenge applications with a large number of discrete latent variables and mixed-type observations. To address these challenges, we propose discrete causal representation learning (DCRL), a generative framework that models a directed acyclic graph among discrete latent variables, along with a sparse bipartite graph linking latent and observed layers. This design accommodates continuous, count, and binary responses through flexible measurement models while maintaining interpretability. Under mild conditions, we prove that both the bipartite measurement graph and the latent causal graph are identifiable from the observed data distribution alone. We further propose a three-stage estimate-resample-discovery pipeline: penalized estimation of the generative model parameters, resampling of latent configurations from the fitted model, and score-based causal discovery on the resampled latents. We establish the consistency of this procedure, ensuring reliable recovery of the latent causal structure. Empirical studies on educational assessment and synthetic image data demonstrate that DCRL recovers sparse and interpretable latent causal structures.