Efficient Generative Prediction for EHR Foundation Models: The SCOPE and REACH Estimators

TL;DR

This paper introduces two novel estimators, SCOPE and REACH, that improve efficiency and accuracy in generative EHR models by reducing variance and computational cost during outcome prediction.

Contribution

The paper proposes SCOPE and REACH estimators that leverage next-token probabilities, providing unbiased, variance-reducing methods for generative EHR models, with empirical validation on clinical outcomes.

Findings

SCOPE and REACH match 100-sample Monte Carlo accuracy with fewer samples.

They achieve 2.5x to 3.4x token reduction, exceeding 80x for rare outcomes.

Calibration is preserved across all evaluated outcomes.

Abstract

Generative foundation models trained on tokenized electronic health record (EHR) timelines show promise for clinical outcome prediction via Monte Carlo sampling of simulated future trajectories. However, this approach suffers from three coupled limitations: sparse estimate distributions that poorly differentiate patient risk levels, extreme computational cost, and high sampling variance. We propose two new estimators that leverage next-token probability distributions underutilized by standard Monte Carlo: the Sum of Conditional Outcome Probability Estimator (SCOPE) and Risk Estimation from Anticipated Conditional Hazards (REACH). We prove both are unbiased, that REACH guarantees variance reduction over Monte Carlo for any model and outcome, and that REACH is a Rao-Blackwellization of any naive importance sampling scheme that preserves the non-outcome token distribution. Empirically, across $11$ clinically important outcomes in MIMIC-IV and the UChicago health system, SCOPE and REACH match $100$-sample Monte Carlo accuracy with median token reductions of $2.5\times$ to $3.4\times$ and reductions exceeding $80\times$ for the rarest outcomes, with calibration preserved throughout. Because SCOPE reuses a single sampled pool across an arbitrary number of outcomes at no marginal generation cost while REACH provides a per-task variance guarantee, the two estimators are complementary in deployment and together meaningfully reduce the inference budget required for generative EHR foundation models, particularly for rare, high-impact outcomes in healthcare.