MATA-Former & SIICU: Semantic Aware Temporal Alignment for High-Fidelity ICU Risk Prediction

TL;DR

This paper introduces MATA-Former, a novel transformer model that uses event semantics for better clinical risk prediction, along with PSL for continuous risk modeling, validated on large ICU datasets.

Contribution

The work presents a new semantic-aware transformer and a soft labeling approach, improving ICU risk prediction from complex clinical data.

Findings

Outperforms existing methods on SIICU and MIMIC-IV datasets.

Effectively models risks over full trajectories with continuous outputs.

Demonstrates robustness and generalization in clinical risk forecasting.

Abstract

Forecasting evolving clinical risks relies on intrinsic pathological dependencies rather than mere chronological proximity, yet current methods struggle with coarse binary supervision and physical timestamps. To align predictive modeling with clinical logic, we propose the Medical-semantics Aware Time-ALiBi Transformer (MATA-Former), utilizing event semantics to dynamically parameterize attention weights to prioritize causal validity over time lags. Furthermore, we introduce Plateau-Gaussian Soft Labeling (PSL), reformulating binary classification into continuous multi-horizon regression for full-trajectory risk modeling. Evaluated on SIICU -- a newly constructed dataset featuring over 506k events with rigorous expert-verified, fine-grained annotations -- and the MIMIC-IV dataset, our framework demonstrates superior efficacy and robust generalization in capturing risks from text-intensive, irregular clinical time series.