Learning Dynamic Representations and Policies from Multimodal Clinical Time-Series with Informative Missingness

TL;DR

This paper introduces a framework for learning patient representations from multimodal clinical time-series data that explicitly leverages informative missingness to improve treatment policy learning and outcome prediction.

Contribution

It proposes a novel multimodal encoder combined with Bayesian filtering to utilize observation patterns, advancing patient state modeling from clinical data.

Findings

Improved offline treatment policy learning with FQE 0.679 versus 0.528.

Enhanced mortality prediction with AUROC 0.886 on MIMIC-III.

Framework effective across multiple ICU cohorts.

Abstract

Multimodal clinical records contain structured measurements and clinical notes recorded over time, offering rich temporal information about the evolution of patient health. Yet these observations are sparse, and whether they are recorded depends on the patient's latent condition. Observation patterns also differ across modalities, as structured measurements and clinical notes arise under distinct recording processes. While prior work has developed methods that accommodate missingness in clinical time series, how to extract and use the information carried by the observation process itself remains underexplored. We therefore propose a patient representation learning framework for multimodal clinical time series that explicitly leverages informative missingness. The framework combines (1) a multimodal encoder that captures signals from structured and textual data together with their observation patterns, (2) a Bayesian filtering module that updates a latent patient state over time from observed multimodal signals, and (3) downstream modules for offline treatment policy learning and patient outcome prediction based on the learned patient state. We evaluate the framework on ICU sepsis cohorts from MIMIC-III, MIMIC-IV, and eICU. It improves both offline treatment policy learning and adverse outcome prediction, achieving FQE 0.679 versus 0.528 for clinician behavior and AUROC 0.886 for post-72-hour mortality prediction on MIMIC-III.