Intercept Cancer: Cancer Pre-Screening with Large Scale Healthcare Foundation Models

TL;DR

This paper introduces CATCH-FM, a large-scale healthcare foundation model trained on electronic health records to pre-screen for cancer risk, enabling early detection with high accuracy and robustness across diverse populations.

Contribution

The paper develops and pretrains large EHR foundation models up to 2.4 billion parameters, demonstrating their effectiveness in cancer risk prediction from medical records.

Findings

Achieves 50% sensitivity at 99% specificity in predicting first cancer risks.

Outperforms feature-based models and general medical LLMs by up to 20% AUPRC.

Demonstrates robustness across diverse patient demographics and healthcare systems.

Abstract

Cancer screening, leading to early detection, saves lives. Unfortunately, existing screening techniques require expensive and intrusive medical procedures, not globally available, resulting in too many lost would-be-saved lives. We present CATCH-FM, CATch Cancer early with Healthcare Foundation Models, a cancer pre-screening methodology that identifies high-risk patients for further screening solely based on their historical medical records. With millions of electronic healthcare records (EHR), we establish the scaling law of EHR foundation models pretrained on medical code sequences, pretrain compute-optimal foundation models of up to 2.4 billion parameters, and finetune them on clinician-curated cancer risk prediction cohorts. In our retrospective evaluation comprising of thirty thousand patients, CATCH-FM achieves strong efficacy, with 50% sensitivity in predicting first cancer risks at 99% specificity cutoff, and outperforming feature-based tree models and both general and medical LLMs by up to 20% AUPRC. Despite significant demographic, healthcare system, and EHR coding differences, CATCH-FM achieves state-of-the-art pancreatic cancer risk prediction on the EHRSHOT few-shot leaderboard, outperforming EHR foundation models pretrained using on-site patient data. Our analysis demonstrates the robustness of CATCH-FM in various patient distributions, the benefits of operating in the ICD code space, and its ability to capture non-trivial cancer risk factors. Our code will be open-sourced.