LLM-Driven Treatment Effect Estimation Under Inference Time Text Confounding

TL;DR

This paper addresses the challenge of estimating treatment effects in medicine when inference relies on incomplete textual patient data, proposing a novel LLM-based framework to mitigate bias caused by inference time text confounding.

Contribution

It formalizes the inference time text confounding problem and introduces a new framework combining large language models with a doubly robust learner to improve treatment effect estimation.

Findings

Framework effectively reduces bias in treatment effect estimates.

Experimental results show improved accuracy over baseline methods.

Demonstrates applicability in real-world clinical scenarios.

Abstract

Estimating treatment effects is crucial for personalized decision-making in medicine, but this task faces unique challenges in clinical practice. At training time, models for estimating treatment effects are typically trained on well-structured medical datasets that contain detailed patient information. However, at inference time, predictions are often made using textual descriptions (e.g., descriptions with self-reported symptoms), which are incomplete representations of the original patient information. In this work, we make three contributions. (1) We show that the discrepancy between the data available during training time and inference time can lead to biased estimates of treatment effects. We formalize this issue as an inference time text confounding problem, where confounders are fully observed during training time but only partially available through text at inference time. (2) To address this problem, we propose a novel framework for estimating treatment effects that explicitly accounts for inference time text confounding. Our framework leverages large language models together with a custom doubly robust learner to mitigate biases caused by the inference time text confounding. (3) Through a series of experiments, we demonstrate the effectiveness of our framework in real-world applications.