Modeling Patient Care Trajectories with Transformer Hawkes Processes

TL;DR

This paper introduces a Transformer Hawkes Process model for continuous-time patient care trajectories, effectively capturing event dependencies and addressing class imbalance to improve prediction of future healthcare events.

Contribution

It combines Transformer encoding with Hawkes processes and introduces an imbalance-aware training strategy for better modeling of irregular healthcare events.

Findings

Improved prediction accuracy for healthcare event types and timings.

Enhanced sensitivity to rare but critical events.

Clinically meaningful insights into high-risk patient populations.

Abstract

Patient healthcare utilization consists of irregularly time-stamped events, such as outpatient visits, inpatient admissions, and emergency encounters, forming individualized care trajectories. Modeling these trajectories is crucial for understanding utilization patterns and predicting future care needs, but is challenging due to temporal irregularity and severe class imbalance. In this work, we build on the Transformer Hawkes Process framework to model patient trajectories in continuous time. By combining Transformer-based history encoding with Hawkes process dynamics, the model captures event dependencies and jointly predicts event type and time-to-event. To address extreme imbalance, we introduce an imbalance-aware training strategy using inverse square-root class weighting. This improves sensitivity to rare but clinically important events without altering the data distribution. Experiments on real-world data demonstrate improved performance and provide clinically meaningful insights for identifying high-risk patient populations.