Building Reality Checks into the Translational Pathway for Diagnostic and Prognostic Models

TL;DR

This paper emphasizes the importance of rigorous external validation for diagnostic and prognostic models and proposes a framework to improve the characterization of validation cohorts to enhance model transferability.

Contribution

It introduces a structured approach to incorporate real-world validation checks into the translational process for medical models.

Findings

Only 5.6% of models included external validation.

Many validation studies lack sufficient documentation.

Proposes key steps to improve model transferability.

Abstract

There has been a significant increase in the number of diagnostic and prognostic models published in the last decade. Testing such models in an independent, external validation cohort gives some assurance the model will transfer to a naturalistic, healthcare setting. Of 2,147 published models in the PubMed database, we found just 120 included some kind of separate external validation cohort. Of these studies not all were sufficiently well documented to allow a judgement about whether that model was likely to transfer to other centres, with other patients, treated by other clinicians, using data scored or analysed by other laboratories. We offer a solution to better characterizing the validation cohort and identify the key steps on the translational pathway for diagnostic and prognostic models.