Automated Text Mining of Experimental Methodologies from Biomedical Literature

TL;DR

This paper introduces a fine-tuned DistilBERT model for classifying biomedical texts, demonstrating improved accuracy and efficiency over traditional methods, with potential applications across research industries.

Contribution

The work develops a specialized, pre-trained DistilBERT model for biomedical text classification, outperforming traditional RNN and LSTM approaches in accuracy and speed.

Findings

Model reduced size by 40% and was 60% faster.

Outperformed traditional RNN and LSTM classification methods.

Proven effectiveness in biomedical literature classification.

Abstract

Biomedical literature is a rapidly expanding field of science and technology. Classification of biomedical texts is an essential part of biomedicine research, especially in the field of biology. This work proposes the fine-tuned DistilBERT, a methodology-specific, pre-trained generative classification language model for mining biomedicine texts. The model has proven its effectiveness in linguistic understanding capabilities and has reduced the size of BERT models by 40\% but by 60\% faster. The main objective of this project is to improve the model and assess the performance of the model compared to the non-fine-tuned model. We used DistilBert as a support model and pre-trained on a corpus of 32,000 abstracts and complete text articles; our results were impressive and surpassed those of traditional literature classification methods by using RNN or LSTM. Our aim is to integrate this highly specialised and specific model into different research industries.