The Crisis Of Evidence: Why Probability And Statistics Cannot Discover Cause

TL;DR

This paper argues that probability and statistical models are fundamentally incapable of identifying causation, emphasizing their proper use for uncertainty quantification rather than causal inference, and criticizes common practices like hypothesis testing and reporting relative risk.

Contribution

It critically examines the limitations of classical statistical methods in causal inference and advocates for abandoning hypothesis tests and Bayes factors in causal analysis.

Findings

Probability models explain uncertainty, not causation.

Hypothesis tests and Bayes factors should be discarded.

Reporting relative risk overstates certainty.

Abstract

Probability models are only useful at explaining the uncertainty of what we do not know, and should never be used to say what we already know. Probability and statistical models are useless at discerning cause. Classical statistical procedures, in both their frequentist and Bayesian implementations are, falsely imply they can speak about cause. No hypothesis test, or Bayes factor, should ever be used again. Even assuming we know the cause or partial cause for some set of observations, reporting via relative risk exagerates the certainty we have in the future, often by a lot. This over-certainty is made much worse when parametetric and not predictive methods are used. Unfortunately, predictive methods are rarely used; and even when they are, cause must still be an assumption, meaning (again) certainty in our scientific pronouncements is too high.