Impossibility results for equating the Youden Index with average scoring rules and Tjur $R^2$-like metrics

TL;DR

This paper proves that the Youden index cannot be exactly matched by any average scoring rule or binary outcome metric, emphasizing the fundamental differences in their roles in diagnostic evaluation.

Contribution

It establishes impossibility results demonstrating the fundamental incompatibility between the Youden index and average scoring or outcome-based metrics in diagnostic assessment.

Findings

Youden index cannot equal any average of a scoring rule.

Youden index cannot match any metric averaging over binary outcomes.

Highlights the distinct roles of these metrics in diagnostics.

Abstract

We consider the Youden index fas well as measures evaluating predicted probabilities for the maximum-likelihood estimate of a logistic regression model with predictor the classifier. We give impossibility results showing that the Youden index can not equal any average of a real scoring rule nor any metric averaging over binary outcomes (0s and 1s) for any continuous real-valued scoring rule. This shows the obstructions of such potential equivalences and highlights the distinct roles these metrics play in diagnostic assessment.