Possibility-theoretic statistical inference offers performance and probativeness assurances

TL;DR

This paper introduces valid inferential models (IMs) that provide both performance guarantees and reliable hypothesis probing, addressing concerns raised by the replication crisis and enhancing scientific confidence.

Contribution

The paper demonstrates that inferential models (IMs) can simultaneously achieve performance and probativeness, with a new result ensuring reliable hypothesis probing.

Findings

IMs achieve both performance and probativeness properties

A new result guarantees the reliability of IM's probing

Comparison with Mayo's severe testing framework highlights differences

Abstract

Statisticians are largely focused on developing methods that perform well in a frequentist sense -- even the Bayesians. But the widely-publicized replication crisis suggests that these performance guarantees alone are not enough to instill confidence in scientific discoveries. In addition to reliably detecting hypotheses that are (in)compatible with data, investigators require methods that can probe for hypotheses that are actually supported by the data. In this paper, we demonstrate that valid inferential models (IMs) achieve both performance and probativeness properties and we offer a powerful new result that ensures the IM's probing is reliable. We also compare and contrast the IM's dual performance and probativeness abilities with that of Deborah Mayo's severe testing framework.