Semi-tail Units: A Universal Scale for Test Statistics and Efficiency

TL;DR

This paper introduces semi-tail units, a universal, intuitive scale for test statistics that standardizes tail probabilities, simplifies interpretation, and enhances efficiency measurement across diverse hypothesis tests.

Contribution

It proposes $z$- and $s$-values as new quantile-based measures that unify and simplify the interpretation of test statistics on a common scale.

Findings

Semi-tail units provide an intuitive logarithmic scale for tail probabilities.

Critical values follow simple arithmetic progressions in semi-tail units.

Adding $s$-values from independent studies combines evidence naturally.

Abstract

We introduce $\zeta$- and $s$-values as quantile-based standardizations that are particularly suited for hypothesis testing. Unlike p-values, which express tail probabilities, $s$-values measure the number of semi-tail units into a distribution's tail, where each unit represents a halving of the tail area. This logarithmic scale provides intuitive interpretation: $s=3.3$ corresponds to the 10th percentile, $s=4.3$ to the 5th percentile, and $s=5.3$ to the 2.5th percentile. For two-tailed tests, $\zeta$-values extend this concept symmetrically around the median. We demonstrate how these measures unify the interpretation of all test statistics on a common scale, eliminating the need for distribution-specific tables. The approach offers practical advantages: critical values follow simple arithmetic progressions, combining evidence from independent studies reduces to the addition of $s$-values, and semi-tail units provide the natural scale for expressing Bahadur slopes. This leads to a new asymptotic efficiency measure based on differences rather than ratios of slopes, where a difference of 0.15 semi-tail units means that the more efficient test moves samples 10\% farther into the tail. Through examples ranging from standardized test scores to poker hand rankings, we show how semi-tail units provide a natural and interpretable scale for quantifying extremeness in any ordered distribution.