A Threshold-free Prospective Prediction Accuracy Measure for Censored Time to Event Data

TL;DR

This paper introduces a new threshold-free metric for evaluating the prospective predictive accuracy of risk scores in censored time-to-event data, avoiding subjective cutoff choices and improving clinical risk assessment.

Contribution

It proposes a novel nonparametric, threshold-free summary index for positive predictive values that accounts for time-dependent event status and provides inference methods for comparing risk scores.

Findings

The new measure performs well in finite-sample simulations.

It effectively compares risk scores without requiring a cutoff threshold.

Application to real data demonstrates practical utility.

Abstract

Prediction performance of a risk scoring system needs to be carefully assessed before its adoption in clinical practice. Clinical preventive care often uses risk scores to screen asymptomatic population. The primary clinical interest is to predict the risk of having an event by a pre-specified future time $t_0$. Prospective accuracy measures such as positive predictive values have been recommended for evaluating the predictive performance. However, for commonly used continuous or ordinal risk score systems, these measures require a subjective cutoff threshold value that dichotomizes the risk scores. The need for a cut-off value created barriers for practitioners and researchers. In this paper, we propose a threshold-free summary index of positive predictive values that accommodates time-dependent event status. We develop a nonparametric estimator and provide an inference procedure for comparing this summary measure between competing risk scores for censored time to event data. We conduct a simulation study to examine the finite-sample performance of the proposed estimation and inference procedures. Lastly, we illustrate the use of this measure on a real data example, comparing two risk score systems for predicting heart failure in childhood cancer survivors.