Using large language models for sensitivity analysis in causal inference: case studies on Cornfield inequality and E-value

TL;DR

This study evaluates the effectiveness of large language models in conducting sensitivity analyses for observational studies, focusing on Cornfield inequalities and E-values, and finds that some models can accurately assist in these assessments.

Contribution

First investigation into using LLMs for sensitivity analysis, demonstrating their potential to accurately reproduce E-values and identify plausible unmeasured confounders.

Findings

ChatGPT, Claude, and Gemini accurately reproduce E-values.

DeepSeek shows small biases in E-value calculation.

All models identify plausible unmeasured confounders.

Abstract

Sensitivity analysis methods such as the Cornfield inequality and the E-value were developed to assess the robustness of observed associations against unmeasured confounding -- a major challenge in observational studies. However, the calculation and interpretation of these methods can be difficult for clinicians and interdisciplinary researchers. Recent advances in large language models (LLMs) offer accessible tools that could assist sensitivity analyses, but their reliability in this context has not been studied. We assess four widely used LLMs, ChatGPT, Claude, DeepSeek, and Gemini, on their ability to conduct sensitivity analyses using Cornfield inequalities and E-values. We first extract study-specific information (exposures, outcomes, measured confounders, and effect estimates) from four published observational studies in different fields. Using such information, we develop structured prompts to assess the performance of the LLMs in three aspects: (1) accuracy of E-value calculation, (2) qualitative interpretation of robustness to unmeasured confounding, and (3) suggestion of possible unmeasured confounders. To our knowledge, there has been little prior work on using LLMs for sensitivity analysis, and this study is an early investigation in this area. The results show that ChatGPT, Claude, and Gemini accurately reproduce the E-values, whereas DeepSeek shows small biases. Qualitative conclusions from all the LLMs align with the magnitude of the E-values and the reported effect sizes, and all models identify biologically and epidemiologically plausible unmeasured confounders. These findings suggest that, when guided by structured prompts, LLMs can effectively assist in evaluating unmeasured confounding, and thereby can support study design and decision-making in observational studies.