Contrastive representations of high-dimensional, structured treatments

TL;DR

This paper introduces a contrastive learning method to create unbiased, structured representations of high-dimensional treatments like text or video for causal effect estimation, addressing biases from naive structure use.

Contribution

It proposes a novel contrastive approach that identifies causal factors in high-dimensional treatments and proves unbiased causal effect estimation using these representations.

Findings

The method accurately identifies causal factors in synthetic datasets.

It achieves unbiased causal effect estimates in real-world data.

Benchmark results show improved performance over existing methods.

Abstract

Estimating causal effects is vital for decision making. In standard causal effect estimation, treatments are usually binary- or continuous-valued. However, in many important real-world settings, treatments can be structured, high-dimensional objects, such as text, video, or audio. This provides a challenge to traditional causal effect estimation. While leveraging the shared structure across different treatments can help generalize to unseen treatments at test time, we show in this paper that using such structure blindly can lead to biased causal effect estimation. We address this challenge by devising a novel contrastive approach to learn a representation of the high-dimensional treatments, and prove that it identifies underlying causal factors and discards non-causally relevant factors. We prove that this treatment representation leads to unbiased estimates of the causal effect, and empirically validate and benchmark our results on synthetic and real-world datasets.