Intent Detection and Entity Extraction from BioMedical Literature

TL;DR

This paper evaluates the effectiveness of supervised fine-tuned biomedical transformer models versus general-purpose large-language models in intent detection and entity extraction tasks within biomedical literature, demonstrating the continued relevance of domain-specific models.

Contribution

It provides a comprehensive empirical comparison showing supervised biomedical transformers outperform general LLMs in biomedical NER and intent detection tasks.

Findings

Biomedical transformers like PubMedBERT outperform ChatGPT with few examples.

Supervised fine-tuning remains effective for biomedical NLP tasks.

General-purpose LLMs are less effective than domain-specific models in this context.

Abstract

Biomedical queries have become increasingly prevalent in web searches, reflecting the growing interest in accessing biomedical literature. Despite recent research on large-language models (LLMs) motivated by endeavours to attain generalized intelligence, their efficacy in replacing task and domain-specific natural language understanding approaches remains questionable. In this paper, we address this question by conducting a comprehensive empirical evaluation of intent detection and named entity recognition (NER) tasks from biomedical text. We show that Supervised Fine Tuned approaches are still relevant and more effective than general-purpose LLMs. Biomedical transformer models such as PubMedBERT can surpass ChatGPT on NER task with only 5 supervised examples.