From Text to Treatment Effects: A Meta-Learning Approach to Handling Text-Based Confounding

TL;DR

This paper explores how meta-learning methods can estimate treatment effects from observational data when confounders are expressed as text, showing that pre-trained text representations improve estimates but have limitations compared to perfect confounder knowledge.

Contribution

It introduces a framework for incorporating pre-trained text embeddings into meta-learners for causal inference, highlighting their benefits and current limitations.

Findings

Text-based confounders improve CATE estimation with sufficient data.

Pre-trained text representations outperform tabular-only models.

Limitations exist due to entangled text embeddings.

Abstract

One of the central goals of causal machine learning is the accurate estimation of heterogeneous treatment effects from observational data. In recent years, meta-learning has emerged as a flexible, model-agnostic paradigm for estimating conditional average treatment effects (CATE) using any supervised model. This paper examines the performance of meta-learners when the confounding variables are expressed in text. Through synthetic data experiments, we show that learners using pre-trained text representations of confounders, in addition to tabular background variables, achieve improved CATE estimates compared to those relying solely on the tabular variables, particularly when sufficient data is available. However, due to the entangled nature of the text embeddings, these models do not fully match the performance of meta-learners with perfect confounder knowledge. These findings highlight both the potential and the limitations of pre-trained text representations for causal inference and open up interesting avenues for future research.