GRU-SCANET: unleashing the power of GRU-based sinusoidal capture network for precision-driven named entity recognition

TL;DR

GRU-SCANET is a new model for biomedical named entity recognition that improves precision and efficiency compared to existing methods.

Contribution

GRU-SCANET introduces a novel architecture combining GRUs, attention, and CRF for efficient and precise biomedical NER.

Findings

GRU-SCANET outperformed BioBERT, PubMedBERT, and SOTA models in 8/8 evaluations.

The model effectively handles unbalanced data across multiple biomedical corpora.

It provides a computationally efficient alternative to pre-trained language models for NER.

Abstract

Pre-trained Language Models (PLMs) have achieved remarkable performance across various natural language processing tasks. However, they encounter challenges in biomedical named entity recognition (NER), such as high computational costs and the need for complex fine-tuning. These limitations hinder the efficient recognition of biological entities, especially within specialized corpora. To address these issues, we introduce GRU-SCANET (Gated Recurrent Unit-based Sinusoidal Capture Network), a novel architecture that directly models the relationship between input tokens and entity classes. Our approach offers a computationally efficient alternative for extracting biological entities by capturing contextual dependencies within biomedical texts.

GRU-SCANET combines positional encoding, bidirectional GRUs (BiGRUs), an attention-based encoder, and a conditional random field (CRF) decoder to achieve high precision in entity labeling. This design effectively mitigates the challenges posed by unbalanced data across multiple corpora. Our model consistently outperforms leading benchmarks, achieving better performance than BioBERT (8/8 evaluations), PubMedBERT (5/5 evaluations), and the previous state-of-the-art (SOTA) models (8/8 evaluations), including Bern2 (5/5 evaluations). These results highlight the strength of our approach in capturing token-entity relationships more effectively than existing methods, advancing the state of biomedicalNER.

https://github.com/ANR-DIG-AI/GRU-SCANET.

Figures