Bangla MedER: Multi-BERT Ensemble Approach for the Recognition of Bangla Medical Entity

TL;DR

This paper introduces a Multi-BERT Ensemble approach for Bangla Medical Entity Recognition, significantly improving accuracy over baseline models and addressing data scarcity in low-resource language NLP.

Contribution

It proposes a novel Multi-BERT Ensemble method and creates a high-quality dataset for Bangla MedER, advancing low-resource medical NLP research.

Findings

Multi-BERT Ensemble achieved 89.58% accuracy

11.80% accuracy improvement over single BERT

Developed a new annotated Bangla MedER dataset

Abstract

Medical Entity Recognition (MedER) is an essential NLP task for extracting meaningful entities from the medical corpus. Nowadays, MedER-based research outcomes can remarkably contribute to the development of automated systems in the medical sector, ultimately enhancing patient care and outcomes. While extensive research has been conducted on MedER in English, low-resource languages like Bangla remain underexplored. Our work aims to bridge this gap. For Bangla medical entity recognition, this study first examined a number of transformer models, including BERT, DistilBERT, ELECTRA, and RoBERTa. We also propose a novel Multi-BERT Ensemble approach that outperformed all baseline models with the highest accuracy of 89.58%. Notably, it provides an 11.80% accuracy improvement over the single-layer BERT model, demonstrating its effectiveness for this task. A major challenge in MedER for low-resource languages is the lack of annotated datasets. To address this issue, we developed a high-quality dataset tailored for the Bangla MedER task. The dataset was used to evaluate the effectiveness of our model through multiple performance metrics, demonstrating its robustness and applicability. Our findings highlight the potential of Multi-BERT Ensemble models in improving MedER for Bangla and set the foundation for further advancements in low-resource medical NLP.