PULSE-ICU: A Pretrained Unified Long-Sequence Encoder for Multi-task Prediction in Intensive Care Units

TL;DR

PULSE-ICU is a self-supervised, unified model that effectively encodes complex ICU data for multiple predictive tasks, demonstrating robustness and improved performance across diverse clinical datasets.

Contribution

The paper introduces PULSE-ICU, a novel foundation model that captures ICU event representations from large-scale EHR data without manual feature engineering, enabling multi-task prediction.

Findings

Achieved strong performance on 18 ICU prediction tasks.

Demonstrated robustness across external datasets with minimal fine-tuning.

Improved data efficiency and adaptability for ICU decision support.

Abstract

Intensive care unit (ICU) data are highly irregular, heterogeneous, and temporally fragmented, posing challenges for generalizable clinical prediction. We present PULSE-ICU, a self-supervised foundation model that learns event-level ICU representations from large-scale EHR sequences without resampling or manual feature engineering. A unified embedding module encodes event identity, continuous values, units, and temporal attributes, while a Longformer-based encoder enables efficient modeling of long trajectories. PULSE-ICU was fine-tuned across 18 prediction tasks, including mortality, intervention forecasting, and phenotype identification, achieving strong performance across task types. External validation on eICU, HiRID, and P12 showed substantial improvements with minimal fine-tuning, demonstrating robustness to domain shift and variable constraints. These findings suggest that foundation-style modeling can improve data efficiency and adaptability, providing a scalable framework for ICU decision support across diverse clinical environments.