Learning Longitudinal Health Representations from EHR and Wearable Data

TL;DR

This paper introduces a multimodal foundation model that jointly learns from electronic health records and wearable data, producing temporally coherent and clinically grounded health representations that improve long-term forecasting and risk modeling.

Contribution

It presents a novel multimodal model combining EHR and wearable data as a continuous time latent process with shared encoders and pretraining objectives, enhancing health representation quality.

Findings

Outperforms EHR-only and wearable-only models on forecasting tasks.

Achieves better performance at long horizons and with missing data.

Produces more faithful longitudinal health representations.

Abstract

Foundation models trained on electronic health records show strong performance on many clinical prediction tasks but are limited by sparse and irregular documentation. Wearable devices provide dense continuous physiological signals but lack semantic grounding. Existing methods usually model these data sources separately or combine them through late fusion. We propose a multimodal foundation model that jointly represents electronic health records and wearable data as a continuous time latent process. The model uses modality specific encoders and a shared temporal backbone pretrained with self supervised and cross modal objectives. This design produces representations that are temporally coherent and clinically grounded. Across forecasting physiological and risk modeling tasks the model outperforms strong electronic health record only and wearable only baselines especially at long horizons and under missing data. These results show that joint electronic health record and wearable pretraining yields more faithful representations of longitudinal health.