ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission

TL;DR

This paper introduces ClinicalBERT, a transformer-based model that effectively captures information from clinical notes, improving hospital readmission predictions beyond traditional structured data methods.

Contribution

The paper develops ClinicalBERT, a novel transformer-based approach that models clinical notes, demonstrating superior performance in predicting 30-day hospital readmissions.

Findings

ClinicalBERT uncovers high-quality relationships between medical concepts.

It outperforms baseline models on readmission prediction tasks.

Code and models are publicly available.

Abstract

Clinical notes contain information about patients that goes beyond structured data like lab values and medications. However, clinical notes have been underused relative to structured data, because notes are high-dimensional and sparse. This work develops and evaluates representations of clinical notes using bidirectional transformers (ClinicalBERT). ClinicalBERT uncovers high-quality relationships between medical concepts as judged by humans. ClinicalBert outperforms baselines on 30-day hospital readmission prediction using both discharge summaries and the first few days of notes in the intensive care unit. Code and model parameters are available.