Testing with p*-values: Between p-values, mid p-values, and e-values

TL;DR

This paper introduces p*-values, a generalized concept encompassing p-values, mid p-values, and e-values, providing a unified framework with multiple interpretations and applications in statistical testing.

Contribution

The paper defines p*-values, explores their interpretations, and develops methods for merging and calibrating them, bridging p-values and e-values with practical testing improvements.

Findings

Unified stochastic representation of p-values, mid p-values, and p*-values

Methods for merging dependent or independent p*-values

Calibrators linking p*-values, p-values, and e-values

Abstract

We introduce the notion of p*-values (p*-variables), which generalizes p-values (p-variables) in several senses. The new notion has four natural interpretations: operational, probabilistic, Bayesian, and frequentist. A main example of a p*-value is a mid p-value, which arises in the presence of discrete test statistics. A unified stochastic representation for p-values, mid p-values, and p*-values is obtained to illustrate the relationship between the three objects. We study several ways of merging arbitrarily dependent or independent p*-values into one p-value or p*-value. Admissible calibrators of p*-values to and from p-values and e-values are obtained with nice mathematical forms, revealing the role of p*-values as a bridge between p-values and e-values. The notion of p*-values becomes useful in many situations even if one is only interested in p-values, mid p-values, or e-values. In particular, deterministic tests based on p*-values can be applied to improve some classic methods for p-values and e-values.