Theoretical Justification of the Bi Error Method

TL;DR

This paper introduces the Bi Error method, a new statistical decision framework based on the Youden Index, designed to replace traditional p-value thresholds and improve scientific decision-making with greater flexibility and accessibility.

Contribution

It provides a theoretically justified, p-value-free 5-step decision method using the Youden Index, emphasizing contextual error analysis and statistical power.

Findings

The Bi Error method allows neutral results and does not rely on default thresholds.

It improves accessibility by eliminating the need for p-values.

The method is supported by a generalized analytical derivation of the Youden Index.

Abstract

Incorrect usage of $p$-values, particularly within the context of significance testing using the arbitrary .05 threshold, has become a major problem in modern statistical practice. The prevalence of this problem can be traced back to the context-free 5-step method commonly taught to undergraduates: we teach it because it is what is done, and we do it because it is what we are taught. This hold particularly true for practitioners of statistics who are not formal statisticians. Thus, in order to improve scientific practice and overcome statistical dichotomania, an accessible replacement for the 5-step method is warranted. We propose a method foundational on the utilization of the Youden Index as a potential decision threshold, which has been shown in the literature to be effective in conjunction with neutral zones. Unlike the traditional 5-step method, our 5-step method (the Bi Error method) allows for neutral results, does not require $p$-values, and does not provide any default threshold values. Instead, our method explicitly requires contextual error analysis as well as quantification of statistical power. Furthermore, and in part due to its lack of usage of p-values, the method sports improved accessibility. This accessibility is supported by a generalized analytical derivation of the Youden Index.