Measurement of statistical evidence on an absolute scale following thermodynamic principles

TL;DR

This paper introduces a thermodynamic framework for measuring statistical evidence on an absolute scale, addressing limitations of traditional statistics like p-values and enabling consistent comparison across studies.

Contribution

It proposes a novel thermodynamic-based method to quantify statistical evidence on an absolute scale, improving calibration and interpretability of evidence measures.

Findings

Developed a thermodynamic model for evidence measurement

Demonstrated consistent evidence calibration across studies

Connected statistical evidence with physical thermodynamic principles

Abstract

Statistical analysis is used throughout biomedical research and elsewhere to assess strength of evidence. We have previously argued that typical outcome statistics (including p-values and maximum likelihood ratios) have poor measure-theoretic properties: they can erroneously indicate decreasing evidence as data supporting an hypothesis accumulate; and they are not amenable to calibration, necessary for meaningful comparison of evidence across different study designs, data types, and levels of analysis. We have also previously proposed that thermodynamic theory, which allowed for the first time derivation of an absolute measurement scale for temperature (T), could be used to derive an absolute scale for evidence (E). Here we present a novel thermodynamically-based framework in which measurement of E on an absolute scale, for which "one degree" always means the same thing, becomes possible for the first time. The new framework invites us to think about statistical analyses in terms of the flow of (evidential) information, placing this work in the context of a growing literature on connections among physics, information theory, and statistics.