Temporal Supervised Contrastive Learning for Modeling Patient Risk Progression

TL;DR

This paper introduces a supervised contrastive learning framework tailored for modeling patient risk progression over time, effectively capturing temporal dynamics and improving prediction accuracy in clinical datasets.

Contribution

The paper proposes a novel temporal supervised contrastive learning method that incorporates a nearest neighbor pairing mechanism to better model patient data over time.

Findings

Outperforms state-of-the-art baselines in mortality prediction and disease progression tracking.

Recovers the correct embedding structure in synthetic datasets.

Nearest neighbor pairing is crucial for model performance.

Abstract

We consider the problem of predicting how the likelihood of an outcome of interest for a patient changes over time as we observe more of the patient data. To solve this problem, we propose a supervised contrastive learning framework that learns an embedding representation for each time step of a patient time series. Our framework learns the embedding space to have the following properties: (1) nearby points in the embedding space have similar predicted class probabilities, (2) adjacent time steps of the same time series map to nearby points in the embedding space, and (3) time steps with very different raw feature vectors map to far apart regions of the embedding space. To achieve property (3), we employ a nearest neighbor pairing mechanism in the raw feature space. This mechanism also serves as an alternative to data augmentation, a key ingredient of contrastive learning, which lacks a standard procedure that is adequately realistic for clinical tabular data, to our knowledge. We demonstrate that our approach outperforms state-of-the-art baselines in predicting mortality of septic patients (MIMIC-III dataset) and tracking progression of cognitive impairment (ADNI dataset). Our method also consistently recovers the correct synthetic dataset embedding structure across experiments, a feat not achieved by baselines. Our ablation experiments show the pivotal role of our nearest neighbor pairing.