Veridical Data Science for Medical Foundation Models

TL;DR

This paper critically examines how the rise of foundation models in medicine alters traditional data science workflows, highlighting challenges to veridical data science principles and proposing refined guidelines for responsible use.

Contribution

It analyzes the impact of foundation models on medical data science workflows and offers recommendations to align them with core principles of veridical data science.

Findings

Foundation models shift the data science workflow in medicine.

Current workflows challenge principles of predictability, computability, and stability.

Proposed guidelines aim to improve transparency and reproducibility.

Abstract

The advent of foundation models (FMs) such as large language models (LLMs) has led to a cultural shift in data science, both in medicine and beyond. This shift involves moving away from specialized predictive models trained for specific, well-defined domain questions to generalist FMs pre-trained on vast amounts of unstructured data, which can then be adapted to various clinical tasks and questions. As a result, the standard data science workflow in medicine has been fundamentally altered; the foundation model lifecycle (FMLC) now includes distinct upstream and downstream processes, in which computational resources, model and data access, and decision-making power are distributed among multiple stakeholders. At their core, FMs are fundamentally statistical models, and this new workflow challenges the principles of Veridical Data Science (VDS), hindering the rigorous statistical analysis expected in transparent and scientifically reproducible data science practices. We critically examine the medical FMLC in light of the core principles of VDS: predictability, computability, and stability (PCS), and explain how it deviates from the standard data science workflow. Finally, we propose recommendations for a reimagined medical FMLC that expands and refines the PCS principles for VDS including considering the computational and accessibility constraints inherent to FMs.