EMBRE: Entity-aware Masking for Biomedical Relation Extraction

TL;DR

The paper presents EMBRE, a novel entity-aware masking approach that enhances biomedical relation extraction by pretraining models with entity masking, leading to improved performance in extracting entity pairs, relations, and novelty.

Contribution

Introduces EMBRE, a new entity-aware masking pretraining method that incorporates entity knowledge into neural networks for biomedical relation extraction.

Findings

Improved extraction of entity pairs, relations, and novelty.

Pretraining with entity masking enhances model robustness.

Achieved better performance than baseline models.

Abstract

Information extraction techniques, including named entity recognition (NER) and relation extraction (RE), are crucial in many domains to support making sense of vast amounts of unstructured text data by identifying and connecting relevant information. Such techniques can assist researchers in extracting valuable insights. In this paper, we introduce the Entity-aware Masking for Biomedical Relation Extraction (EMBRE) method for biomedical relation extraction, as applied in the context of the BioRED challenge Task 1, in which human-annotated entities are provided as input. Specifically, we integrate entity knowledge into a deep neural network by pretraining the backbone model with an entity masking objective. We randomly mask named entities for each instance and let the model identify the masked entity along with its type. In this way, the model is capable of learning more specific knowledge and more robust representations. Then, we utilize the pre-trained model as our backbone to encode language representations and feed these representations into two multilayer perceptron (MLPs) to predict the logits for relation and novelty, respectively. The experimental results demonstrate that our proposed method can improve the performances of entity pair, relation and novelty extraction over our baseline.