Quantifying Sufficient Randomness for Causal Inference

TL;DR

This paper introduces a methodology to quantify the amount of randomness needed in data to reliably infer causality from natural experiments, using standardized measures of variation.

Contribution

It presents a novel sensitivity analysis framework that characterizes uncertainty in causal inference through standardized randomness parameters and a computable threshold.

Findings

Derived a threshold for sufficient randomness in data for causal inference

Provided a method to compute this threshold from contingency table data

Showed that exceeding this threshold justifies causal conclusions

Abstract

Spurious association arises from covariance between propensity for the treatment and individual risk for the outcome. For sensitivity analysis with stochastic counterfactuals we introduce a methodology to characterize uncertainty in causal inference from natural experiments and quasi-experiments. Our sensitivity parameters are standardized measures of variation in propensity and individual risk, and one minus their geometric mean is an intuitive measure of randomness in the data generating process. Within our latent propensity-risk model, we show how to compute from contingency table data a threshold, $T$, of sufficient randomness for causal inference. If the actual randomness of the data generating process exceeds this threshold then causal inference is warranted.