Multi-Domain Empirical Bayes for Linearly-Mixed Causal Representations

TL;DR

This paper introduces an empirical Bayes approach for causal representation learning across multiple domains, leveraging invariant structures and interventions to improve estimation accuracy in linear models.

Contribution

It develops an EB $f$-modeling algorithm with an EM-style method for causal estimation, enhancing multi-domain CRL performance over existing techniques.

Findings

Achieves more accurate causal variable estimation on synthetic data.

Utilizes invariant structures across domains for improved learning.

Proposes an EM-style algorithm based on causally structured score matching.

Abstract

Causal representation learning (CRL) aims to learn low-dimensional causal latent variables from high-dimensional observations. While identifiability has been extensively studied for CRL, estimation has been less explored. In this paper, we explore the use of empirical Bayes (EB) to estimate causal representations. In particular, we consider the problem of learning from data from multiple domains, where differences between domains are modeled by interventions in a shared underlying causal model. Multi-domain CRL naturally poses a simultaneous inference problem that EB is designed to tackle. Here, we propose an EB $f$-modeling algorithm that improves the quality of learned causal variables by exploiting invariant structure within and across domains. Specifically, we consider a linear measurement model and interventional priors arising from a shared acyclic SCM. When the graph and intervention targets are known, we develop an EM-style algorithm based on causally structured score matching. We further discuss EB $g$-modeling in the context of existing CRL approaches. In experiments on synthetic data, our proposed method achieves more accurate estimation than other methods for CRL.