Novel decision-theoretic and risk-stratification metrics of predictive performance: Application to deciding who should undergo genetic testing

TL;DR

This paper introduces new decision-theoretic and risk-stratification metrics, MRS and NBI, to evaluate predictive models for BRCA1/2 testing, aiming to optimize testing strategies and resource allocation.

Contribution

It develops MRS and NBI metrics that link risk thresholds with model informativeness, providing a decision-theoretic framework for evaluating predictive performance.

Findings

MRS and NBI identify optimal risk thresholds for genetic testing.

The metrics connect traditional measures like AUC to decision-theoretic principles.

Application to BRCA1/2 testing demonstrates practical utility.

Abstract

Currently, women are referred for BRCA1/2 mutation-testing only if their family-history of breast/ovarian cancer implies that their risk of carrying a mutation exceeds 10\%. However, as mutation-testing costs fall, prominent voices have called for testing all women, which would strain clinical resources by testing millions of women, almost all of whom will test negative. To better evaluate risk-thresholds for BRCA1/2 testing, we introduce two broadly applicable, linked metrics: Mean Risk Stratification (MRS) and a decision-theoretic metric, Net Benefit of Information (NBI). MRS and NBI provide a range of risk thresholds at which a marker/model is "optimally informative", in the sense of maximizing both MRS and NBI. NBI is a function of only MRS and the risk-threshold for action, connecting decision-theory to risk-stratification and providing a decision-theoretic rationale for MRS. AUC and Youden's index reflect on both the fraction of maximum MRS, and of maximum NBI, attained by the marker/model, providing AUC and Youden's index with long-sought decision-theoretic and risk-stratification rationale. To evaluate risk-thresholds for BRCA1/2 testing, we propose an eclectic approach considering AUC, Net Benefit, and MRS/NBI. MRS/NBI interpret AUC in the context of mutation-prevalence and provide a range of risk thresholds for which the risk model is optimally informative.