Significance Tests in Climate Science

TL;DR

This paper critiques the misuse of significance tests in climate science, arguing they rarely provide meaningful quantitative confidence and proposing a Bayesian perspective to clarify their interpretation.

Contribution

It introduces a Bayesian analysis to explain significance tests and highlights their limited usefulness in climate research.

Findings

Significance tests are often misused in climate literature.

They rarely offer meaningful quantitative confidence.

Bayesian analysis clarifies the interpretation of significance tests.

Abstract

A large fraction of papers in the climate literature includes erroneous uses of significance tests. A Bayesian analysis is presented to highlight the meaning of significance tests and why typical misuse occurs. It is concluded that a significance test very rarely provides useful quantitative information. The significance statistic is not a quantitative measure of how confident we can be of the 'reality' of a given result.