BioMNER: A Dataset for Biomedical Method Entity Recognition

TL;DR

This paper introduces a new dataset for biomedical method entity recognition, demonstrating that smaller language models combined with CRF outperform larger models in this specialized task.

Contribution

The study presents a novel biomedical method NER dataset and shows that a small ALBERT model with CRF achieves state-of-the-art results, challenging the effectiveness of large-scale models.

Findings

Large models hinder effective pattern learning for biomedical methods.

Small ALBERT with CRF outperforms larger models.

New dataset facilitates future biomedical NER research.

Abstract

Named entity recognition (NER) stands as a fundamental and pivotal task within the realm of Natural Language Processing. Particularly within the domain of Biomedical Method NER, this task presents notable challenges, stemming from the continual influx of domain-specific terminologies in scholarly literature. Current research in Biomedical Method (BioMethod) NER suffers from a scarcity of resources, primarily attributed to the intricate nature of methodological concepts, which necessitate a profound understanding for precise delineation. In this study, we propose a novel dataset for biomedical method entity recognition, employing an automated BioMethod entity recognition and information retrieval system to assist human annotation. Furthermore, we comprehensively explore a range of conventional and contemporary open-domain NER methodologies, including the utilization of cutting-edge large-scale language models (LLMs) customised to our dataset. Our empirical findings reveal that the large parameter counts of language models surprisingly inhibit the effective assimilation of entity extraction patterns pertaining to biomedical methods. Remarkably, the approach, leveraging the modestly sized ALBERT model (only 11MB), in conjunction with conditional random fields (CRF), achieves state-of-the-art (SOTA) performance.