Aim for clinical utility, not just predictive accuracy

TL;DR

This paper emphasizes the importance of focusing on clinical utility rather than just predictive accuracy in prognostic models, proposing a framework to evaluate decision rules using observational data.

Contribution

It introduces a method to formalize and assess prediction-based decision rules through emulating clinical trials with observational data.

Findings

Framework for using observational data to evaluate decision rules

Split-sample approach for model development and validation

Emulation of prediction-driven trials to assess utility

Abstract

The predictions from an accurate prognostic model can be of great interest to patients and clinicians. When predictions are reported to individuals, they may decide to take action to improve their health or they may simply be comforted by the knowledge. However, if there is a clearly defined space of actions in the clinical context, a formal decision rule based on the prediction has the potential to have a much broader impact. Even if it is not the intended use of a developed prediction model, informal decision rules can often be found in practice. The use of a prediction-based decision rule should be formalized and compared to the standard of care in a randomized trial to assess its clinical utility, however, evidence is needed to motivate such a trial. We outline how observational data can be used to propose a decision rule based on a prognostic prediction model. We then propose a framework for emulating a prediction driven trial to evaluate the utility of a prediction-based decision rule in observational data. A split-sample structure can and should be used to develop the prognostic model, define the decision rule, and evaluate its clinical utility.