Bayesian Sensitivity of Causal Inference Estimators under Evidence-Based Priors

TL;DR

This paper introduces Bayesian Sensitivity Value (BSV), a new framework for sensitivity analysis in causal inference that incorporates prior knowledge to assess robustness against assumption violations.

Contribution

It generalizes the s-value framework and proposes BSV to provide more realistic sensitivity assessments using real-world evidence.

Findings

Worst-case sensitivity conclusions can be based on unrealistic assumptions.

BSV incorporates priors from real-world evidence for more realistic sensitivity estimates.

Empirical example demonstrates BSV's applicability in medical observational study.

Abstract

Causal inference, especially in observational studies, relies on untestable assumptions about the true data-generating process. Sensitivity analysis helps us determine how robust our conclusions are when we alter these underlying assumptions. Existing frameworks for sensitivity analysis are concerned with worst-case changes in assumptions. In this work, we argue that using such pessimistic criteria can often become uninformative or lead to conclusions contradicting our prior knowledge about the world. To demonstrate this claim, we generalize the recent s-value framework (Gupta & Rothenh\"ausler, 2023) to estimate the sensitivity of three different common assumptions in causal inference. Empirically, we find that, indeed, worst-case conclusions about sensitivity can rely on unrealistic changes in the data-generating process. To overcome this, we extend the s-value framework with a new sensitivity analysis criterion: Bayesian Sensitivity Value (BSV), which computes the expected sensitivity of an estimate to assumption violations under priors constructed from real-world evidence. We use Monte Carlo approximations to estimate this quantity and illustrate its applicability in an observational study on the effect of diabetes treatments on weight loss.