MeDAL: Medical Abbreviation Disambiguation Dataset for Natural Language Understanding Pretraining

TL;DR

MeDAL is a large medical text dataset created for abbreviation disambiguation, which improves NLP model performance and training efficiency in medical applications.

Contribution

This work introduces MeDAL, a new dataset for medical abbreviation disambiguation, and demonstrates its effectiveness in pre-training models for better downstream medical NLP tasks.

Findings

Pre-training on MeDAL improves model performance on medical tasks.

Pre-training accelerates convergence during fine-tuning.

Models trained on MeDAL outperform baseline models.

Abstract

One of the biggest challenges that prohibit the use of many current NLP methods in clinical settings is the availability of public datasets. In this work, we present MeDAL, a large medical text dataset curated for abbreviation disambiguation, designed for natural language understanding pre-training in the medical domain. We pre-trained several models of common architectures on this dataset and empirically showed that such pre-training leads to improved performance and convergence speed when fine-tuning on downstream medical tasks.