A Causal Ordering Prior for Unsupervised Representation Learning

TL;DR

This paper introduces a novel unsupervised method for representation learning that incorporates causal relationships among latent variables, using a causal ordering prior inspired by functional causal models and Hessian-based loss.

Contribution

It proposes a fully unsupervised approach to causal representation learning that relaxes independence assumptions and enforces causal order in latent space without auxiliary data.

Findings

Demonstrates causal ordering in latent space using Hessian-based loss

Achieves identifiable causal representations without supervision

Extends variational inference to causal latent models

Abstract

Unsupervised representation learning with variational inference relies heavily on independence assumptions over latent variables. Causal representation learning (CRL), however, argues that factors of variation in a dataset are, in fact, causally related. Allowing latent variables to be correlated, as a consequence of causal relationships, is more realistic and generalisable. So far, provably identifiable methods rely on: auxiliary information, weak labels, and interventional or even counterfactual data. Inspired by causal discovery with functional causal models, we propose a fully unsupervised representation learning method that considers a data generation process with a latent additive noise model (ANM). We encourage the latent space to follow a causal ordering via loss function based on the Hessian of the latent distribution.