A Biomedical Pipeline to Detect Clinical and Non-Clinical Named Entities

TL;DR

This paper introduces a machine learning pipeline that enhances biomedical named entity recognition by identifying a broader range of entities, including social determinants of health, and demonstrates superior performance on multiple datasets.

Contribution

The novel pipeline recognizes diverse biomedical entities and incorporates social determinants of health, addressing limitations of previous methods.

Findings

Outperforms baseline methods with around 90% F1 scores on benchmark datasets.

Achieves 95.25% macro-average F1 on curated COVID-19 reports.

Effectively extracts both clinical and non-clinical health-related entities.

Abstract

There are a few challenges related to the task of biomedical named entity recognition, which are: the existing methods consider a fewer number of biomedical entities (e.g., disease, symptom, proteins, genes); and these methods do not consider the social determinants of health (age, gender, employment, race), which are the non-medical factors related to patients' health. We propose a machine learning pipeline that improves on previous efforts in the following ways: first, it recognizes many biomedical entity types other than the standard ones; second, it considers non-clinical factors related to patient's health. This pipeline also consists of stages, such as preprocessing, tokenization, mapping embedding lookup and named entity recognition task to extract biomedical named entities from the free texts. We present a new dataset that we prepare by curating the COVID-19 case reports. The proposed approach outperforms the baseline methods on five benchmark datasets with macro-and micro-average F1 scores around 90, as well as our dataset with a macro-and micro-average F1 score of 95.25 and 93.18 respectively.