Deep Learning Approach for Clinical Risk Identification Using Transformer Modeling of Heterogeneous EHR Data

TL;DR

This paper introduces a Transformer-based model for clinical risk prediction that effectively integrates heterogeneous EHR data, capturing temporal dynamics and semantic importance to improve accuracy and reliability in healthcare decision-making.

Contribution

It presents a novel Transformer architecture with temporal encoding and semantic-weighted pooling tailored for heterogeneous EHR data, enhancing risk classification performance.

Findings

Outperforms traditional machine learning models in accuracy and F1-score

Effectively captures long-term and short-term temporal dependencies

Provides a stable and precise risk identification framework

Abstract

This study proposes a Transformer-based longitudinal modeling method to address challenges in clinical risk classification with heterogeneous Electronic Health Record (EHR) data, including irregular temporal patterns, large modality differences, and complex semantic structures. The method takes multi-source medical features as input and employs a feature embedding layer to achieve a unified representation of structured and unstructured data. A learnable temporal encoding mechanism is introduced to capture dynamic evolution under uneven sampling intervals. The core model adopts a multi-head self-attention structure to perform global dependency modeling on longitudinal sequences, enabling the aggregation of long-term trends and short-term fluctuations across different temporal scales. To enhance semantic representation, a semantic-weighted pooling module is designed to assign adaptive importance to key medical events, improving the discriminative ability of risk-related features. Finally, a linear mapping layer generates individual-level risk scores. Experimental results show that the proposed model outperforms traditional machine learning and temporal deep learning models in accuracy, recall, precision, and F1-Score, achieving stable and precise risk identification in multi-source heterogeneous EHR environments and providing an efficient and reliable framework for clinical intelligent decision-making.