HealthPrompt: A Zero-shot Learning Paradigm for Clinical Natural Language Processing

TL;DR

HealthPrompt introduces a zero-shot learning framework for clinical NLP that leverages prompt-based techniques with pre-trained language models, enabling effective clinical text classification without requiring annotated datasets.

Contribution

The paper presents a novel prompt-based clinical NLP framework, HealthPrompt, demonstrating zero-shot learning capabilities across multiple pre-trained language models in clinical contexts.

Findings

Prompts effectively capture clinical text context

HealthPrompt performs well without training data

Applicable across six different PLMs

Abstract

Deep learning algorithms are dependent on the availability of large-scale annotated clinical text datasets. The lack of such publicly available datasets is the biggest bottleneck for the development of clinical Natural Language Processing(NLP) systems. Zero-Shot Learning(ZSL) refers to the use of deep learning models to classify instances from new classes of which no training data have been seen before. Prompt-based learning is an emerging ZSL technique where we define task-based templates for NLP tasks. We developed a novel prompt-based clinical NLP framework called HealthPrompt and applied the paradigm of prompt-based learning on clinical texts. In this technique, rather than fine-tuning a Pre-trained Language Model(PLM), the task definitions are tuned by defining a prompt template. We performed an in-depth analysis of HealthPrompt on six different PLMs in a no-data setting. Our experiments prove that prompts effectively capture the context of clinical texts and perform remarkably well without any training data.