BioNCERE: Non-Contrastive Enhancement For Relation Extraction In Biomedical Texts

TL;DR

BioNCERE introduces a novel non-contrastive training approach for biomedical relation extraction that reduces annotation costs and achieves near state-of-the-art performance without relying on named entity labels.

Contribution

It presents a new non-contrastive learning method for relation extraction that avoids entity annotations and overfitting, improving efficiency in biomedical text analysis.

Findings

Achieves near state-of-the-art RE performance on SemMedDB

Reduces reliance on named entity annotations

Effectively avoids overfitting and class collapse

Abstract

State-of-the-art models for relation extraction (RE) in the biomedical domain consider finetuning BioBERT using classification, but they may suffer from the anisotropy problem. Contrastive learning methods can reduce this anisotropy phenomena, and also help to avoid class collapse in any classification problem. In the present paper, a new training method called biological non-contrastive relation extraction (BioNCERE) is introduced for relation extraction without using any named entity labels for training to reduce annotation costs. BioNCERE uses transfer learning and non-contrastive learning to avoid full or dimensional collapse as well as bypass overfitting. It resolves RE in three stages by leveraging transfer learning two times. By freezing the weights learned in previous stages in the proposed pipeline and by leveraging non-contrastive learning in the second stage, the model predicts relations without any knowledge of named entities. Experiments have been done on SemMedDB that are almost similar to State-of-the-art performance on RE without using the information of named entities.