Clinical information extraction for Low-resource languages with Few-shot learning using Pre-trained language models and Prompting

TL;DR

This paper evaluates prompt-based few-shot learning with lightweight models for clinical information extraction in low-resource languages, demonstrating significant accuracy improvements and emphasizing interpretability.

Contribution

It provides the first systematic evaluation of prompt-based few-shot learning for clinical text classification in low-resource settings, with detailed interpretability analysis.

Findings

Prompted models with 20 shots outperform traditional models by 30.5% accuracy.

Lightweight, domain-adapted pretrained models are effective for low-resource clinical NLP.

Shapley values validate interpretability and quality of small training datasets.

Abstract

Automatic extraction of medical information from clinical documents poses several challenges: high costs of required clinical expertise, limited interpretability of model predictions, restricted computational resources and privacy regulations. Recent advances in domain-adaptation and prompting methods showed promising results with minimal training data using lightweight masked language models, which are suited for well-established interpretability methods. We are first to present a systematic evaluation of these methods in a low-resource setting, by performing multi-class section classification on German doctor's letters. We conduct extensive class-wise evaluations supported by Shapley values, to validate the quality of our small training data set and to ensure the interpretability of model predictions. We demonstrate that a lightweight, domain-adapted pretrained model, prompted with just 20 shots, outperforms a traditional classification model by 30.5% accuracy. Our results serve as a process-oriented guideline for clinical information extraction projects working with low-resource.