Hierarchical biomarker thresholding: a model-agnostic framework for stability

TL;DR

This paper introduces a model-agnostic hierarchical thresholding framework that improves stability and reproducibility of patient-level biomarker decisions across different sites by accounting for hierarchical dependence and prevalence shifts.

Contribution

It presents a novel risk decomposition theorem and a stability measure for hierarchical thresholding, enhancing decision reproducibility in biomarker pipelines.

Findings

Framework improves decision stability across sites

Decomposition separates fit, shift, and stability contributions

Provides diagnostics like flip-rate and operating-point shift

Abstract

Many biomarker pipelines require patient-level decisions aggregated from instance-level (cell/patch) scores. Thresholds tuned on pooled instances often fail across sites due to hierarchical dependence, prevalence shift, and score-scale mismatch. We present a selection-honest framework for hierarchical thresholding that makes patient-level decisions reproducible and more defensible. At its core is a risk decomposition theorem for selection-honest thresholds. The theorem separates contributions from (i) internal fit and patient-level generalization, (ii) operating-point shift reflecting prevalence and shape changes, and (iii) a stability term that penalizes sensitivity to threshold perturbations. The stability component is computable via patient-block bootstraps mapped through a monotone modulus of risk. This framework is model-agnostic, reconciles heterogeneous decision rules on a quantile scale, and yields monotone-invariant ensembles and reportable diagnostics (e.g. flip-rate, operating-point shift).