Estimating Causal Effects of Text Interventions Leveraging LLMs

TL;DR

This paper introduces CausalDANN, a new method leveraging large language models to estimate causal effects of complex textual interventions, overcoming limitations of traditional methods and enabling analysis of high-dimensional text data in social systems.

Contribution

The paper presents CausalDANN, a novel approach that handles arbitrary textual interventions and domain shifts, advancing causal inference in high-dimensional textual data.

Findings

CausalDANN effectively estimates causal effects from observational text data.

The method demonstrates robustness against domain shifts.

It enables analysis of diverse textual interventions in social systems.

Abstract

Quantifying the effects of textual interventions in social systems, such as reducing anger in social media posts to see its impact on engagement, is challenging. Real-world interventions are often infeasible, necessitating reliance on observational data. Traditional causal inference methods, typically designed for binary or discrete treatments, are inadequate for handling the complex, high-dimensional textual data. This paper addresses these challenges by proposing CausalDANN, a novel approach to estimate causal effects using text transformations facilitated by large language models (LLMs). Unlike existing methods, our approach accommodates arbitrary textual interventions and leverages text-level classifiers with domain adaptation ability to produce robust effect estimates against domain shifts, even when only the control group is observed. This flexibility in handling various text interventions is a key advancement in causal estimation for textual data, offering opportunities to better understand human behaviors and develop effective interventions within social systems.