Causally-Aware Information Bottleneck for Domain Adaptation

TL;DR

This paper introduces a causally-aware information bottleneck framework for domain adaptation, enabling accurate target variable imputation across shifts in causal systems using both linear and nonlinear models.

Contribution

It proposes a novel information bottleneck method tailored for causal domain adaptation, including closed-form solutions for linear models and scalable variational methods for complex data.

Findings

Achieves accurate target variable imputation in synthetic and real datasets.

Provides a unified, lightweight toolkit for causal domain adaptation.

Supports zero-shot deployment to target domains.

Abstract

We tackle a common domain adaptation setting in causal systems. In this setting, the target variable is observed in the source domain but is entirely missing in the target domain. We aim to impute the target variable in the target domain from the remaining observed variables under various shifts. We frame this as learning a compact, mechanism-stable representation. This representation preserves information relevant for predicting the target while discarding spurious variation. For linear Gaussian causal models, we derive a closed-form Gaussian Information Bottleneck (GIB) solution. This solution reduces to a canonical correlation analysis (CCA)-style projection and offers Directed Acyclic Graph (DAG)-aware options when desired. For nonlinear or non-Gaussian data, we introduce a Variational Information Bottleneck (VIB) encoder-predictor. This approach scales to high dimensions and can be trained on source data and deployed zero-shot to the target domain. Across synthetic and real datasets, our approach consistently attains accurate imputations, supporting practical use in high-dimensional causal models and furnishing a unified, lightweight toolkit for causal domain adaptation.