Building Patient Journeys in Hebrew: A Language Model for Clinical Timeline Extraction

TL;DR

This paper introduces a Hebrew medical language model based on DictaBERT 2.0 that extracts structured clinical timelines from electronic health records, aiding in constructing detailed patient journeys while maintaining privacy.

Contribution

The study develops a new Hebrew medical language model with continual pre-training on extensive hospital data and introduces two annotated datasets for evaluating temporal relation extraction.

Findings

Model achieves strong performance on clinical timeline extraction tasks.

Vocabulary adaptation enhances token efficiency.

De-identification does not impair downstream model performance.

Abstract

We present a new Hebrew medical language model designed to extract structured clinical timelines from electronic health records, enabling the construction of patient journeys. Our model is based on DictaBERT 2.0 and continually pre-trained on over five million de-identified hospital records. To evaluate its effectiveness, we introduce two new datasets -- one from internal medicine and emergency departments, and another from oncology -- annotated for event temporal relations. Our results show that our model achieves strong performance on both datasets. We also find that vocabulary adaptation improves token efficiency and that de-identification does not compromise downstream performance, supporting privacy-conscious model development. The model is made available for research use under ethical restrictions.