Knowledge-augmented Pre-trained Language Models for Biomedical Relation Extraction

TL;DR

This study evaluates how different pre-trained language models and contextual data enhancements affect biomedical relation extraction, highlighting the importance of model choice and hyperparameter tuning for optimal performance.

Contribution

It provides a comprehensive evaluation framework for PLMs with contextual information in biomedical RE, including extensive hyperparameter optimization and analysis of external data impact.

Findings

Model choice and hyperparameter tuning are crucial for RE performance.

External contextual data benefits smaller PLMs significantly.

Inclusion of context information yields minor overall improvements.

Abstract

Automatic relationship extraction (RE) from biomedical literature is critical for managing the vast amount of scientific knowledge produced each year. In recent years, utilizing pre-trained language models (PLMs) has become the prevalent approach in RE. Several studies report improved performance when incorporating additional context information while fine-tuning PLMs for RE. However, variations in the PLMs applied, the databases used for augmentation, hyper-parameter optimization, and evaluation methods complicate direct comparisons between studies and raise questions about the generalizability of these findings. Our study addresses this research gap by evaluating PLMs enhanced with contextual information on five datasets spanning four relation scenarios within a consistent evaluation framework. We evaluate three baseline PLMs and first conduct extensive hyperparameter optimization. After selecting the top-performing model, we enhance it with additional data, including textual entity descriptions, relational information from knowledge graphs, and molecular structure encodings. Our findings illustrate the importance of i) the choice of the underlying language model and ii) a comprehensive hyperparameter optimization for achieving strong extraction performance. Although inclusion of context information yield only minor overall improvements, an ablation study reveals substantial benefits for smaller PLMs when such external data was included during fine-tuning.