Learning Robust Intervention Representations with Delta Embeddings

TL;DR

This paper introduces Causal Delta Embeddings, a novel latent space representation for interventions that enhances out-of-distribution robustness in causal image models, without requiring extra supervision.

Contribution

It proposes a new method for learning intervention representations called Causal Delta Embeddings, improving OOD robustness in causal image tasks.

Findings

Causal Delta Embeddings outperform baselines in synthetic benchmarks.

The method achieves significant improvements in real-world OOD scenarios.

It demonstrates effectiveness without additional supervision.

Abstract

Causal representation learning has attracted significant research interest during the past few years, as a means for improving model generalization and robustness. Causal representations of interventional image pairs (also called ``actionable counterfactuals'' in the literature), have the property that only variables corresponding to scene elements affected by the intervention / action are changed between the start state and the end state. While most work in this area has focused on identifying and representing the variables of the scene under a causal model, fewer efforts have focused on representations of the interventions themselves. In this work, we show that an effective strategy for improving out of distribution (OOD) robustness is to focus on the representation of actionable counterfactuals in the latent space. Specifically, we propose that an intervention can be represented by a Causal Delta Embedding that is invariant to the visual scene and sparse in terms of the causal variables it affects. Leveraging this insight, we propose a method for learning causal representations from image pairs, without any additional supervision. Experiments in the Causal Triplet challenge demonstrate that Causal Delta Embeddings are highly effective in OOD settings, significantly exceeding baseline performance in both synthetic and real-world benchmarks.