The continuous net benefit: Assessing the clinical utility of prediction models when informing a continuum of decisions

TL;DR

This paper introduces the continuous net benefit, a novel method for evaluating the clinical utility of prognostic models across multiple decision thresholds, enhancing personalized care and decision-making in medicine.

Contribution

The paper extends the net benefit framework to a continuous version that considers multiple thresholds simultaneously, addressing limitations of existing methods for personalized treatment decisions.

Findings

Demonstrates the continuous net benefit with cardiovascular examples.

Shows improved assessment of model utility across varying thresholds.

Highlights limitations of current decision curve analysis methods.

Abstract

Clinical prognostic models help inform decision-making by estimating a patient's risk of experiencing an outcome in the future. The net benefit is increasingly being used to assess the clinical utility of models. By calculating an appropriately weighted average of the true and false positives of a model, the net benefit assesses the value added by a binary decision policy obtained when thresholding a model. Although such 'treat or not' decisions are common, prognostic models are also often used to tailor and personalise the care of patients, which implicitly involves the consideration of multiple interventions at different risk thresholds. We extend the net benefit to consider multiple decision thresholds simultaneously, by taking a weighted area under a rescaled version of the net benefit curve, deriving the continuous net benefit. In addition to the consideration of a continuum of interventions, we also show how the continuous net benefit can be used for populations with a range of optimal thresholds for a single treatment, due to individual variations in expected treatment benefit or harm, highlighting limitations of current proposed methods that calculate the area under the decision curve. We showcase the continuous net benefit through two examples of cardiovascular preventive care, comparing two modelling choices using the continuous net benefit. The continuous net benefit informs researchers of the clinical utility of models during selection, development, and validation, and helps decision makers understand their usefulness, improving their viability towards implementation.