A New Deep State-Space Analysis Framework for Patient Latent State Estimation and Classification from EHR Time Series Data

TL;DR

This paper introduces a deep state-space analysis framework for modeling and interpreting patient latent states from EHR time series data, aiding disease progression understanding and treatment planning.

Contribution

It presents a novel deep state-space model that captures, visualizes, and clusters patient latent states from EHRs, improving interpretability over existing methods.

Findings

Successfully identified latent states related to prognosis in cancer patients.

Visualized and clustered disease progression and treatment effects.

Outperformed existing methods in capturing interpretable latent space.

Abstract

Many diseases, including cancer and chronic conditions, require extended treatment periods and long-term strategies. Machine learning and AI research focusing on electronic health records (EHRs) have emerged to address this need. Effective treatment strategies involve more than capturing sequential changes in patient test values. It requires an explainable and clinically interpretable model by capturing the patient's internal state over time. In this study, we propose the "deep state-space analysis framework," using time-series unsupervised learning of EHRs with a deep state-space model. This framework enables learning, visualizing, and clustering of temporal changes in patient latent states related to disease progression. We evaluated our framework using time-series laboratory data from 12,695 cancer patients. By estimating latent states, we successfully discover latent states related to prognosis. By visualization and cluster analysis, the temporal transition of patient status and test items during state transitions characteristic of each anticancer drug were identified. Our framework surpasses existing methods in capturing interpretable latent space. It can be expected to enhance our comprehension of disease progression from EHRs, aiding treatment adjustments and prognostic determinations.