Clinical NLP with Attention-Based Deep Learning for Multi-Disease Prediction

TL;DR

This paper introduces an attention-based deep learning model using Transformers for multi-disease prediction from unstructured clinical texts, demonstrating superior performance and robustness on the MIMIC-IV dataset.

Contribution

It proposes a novel Transformer-based architecture with semantic alignment for improved multi-label disease prediction from clinical notes, addressing high-dimensional and unstructured medical text challenges.

Findings

Outperforms existing methods across multiple metrics

Maintains strong generalization under data and noise variations

Effectively captures medical entities and contextual relationships

Abstract

This paper addresses the challenges posed by the unstructured nature and high-dimensional semantic complexity of electronic health record texts. A deep learning method based on attention mechanisms is proposed to achieve unified modeling for information extraction and multi-label disease prediction. The study is conducted on the MIMIC-IV dataset. A Transformer-based architecture is used to perform representation learning over clinical text. Multi-layer self-attention mechanisms are employed to capture key medical entities and their contextual relationships. A Sigmoid-based multi-label classifier is then applied to predict multiple disease labels. The model incorporates a context-aware semantic alignment mechanism, enhancing its representational capacity in typical medical scenarios such as label co-occurrence and sparse information. To comprehensively evaluate model performance, a series of experiments were conducted, including baseline comparisons, hyperparameter sensitivity analysis, data perturbation studies, and noise injection tests. Results demonstrate that the proposed method consistently outperforms representative existing approaches across multiple performance metrics. The model maintains strong generalization under varying data scales, interference levels, and model depth configurations. The framework developed in this study offers an efficient algorithmic foundation for processing real-world clinical texts and presents practical significance for multi-label medical text modeling tasks.