Span-based Joint Entity and Relation Extraction with Transformer Pre-training

TL;DR

This paper presents SpERT, a lightweight attention-based model leveraging BERT for joint entity and relation extraction, emphasizing efficient span reasoning, negative sampling, and localized context, achieving state-of-the-art results.

Contribution

The paper introduces a novel span-based joint extraction model with a localized context approach and strong negative sampling, improving efficiency and accuracy over prior methods.

Findings

Outperforms previous models by up to 2.6% F1 score.

Effective use of BERT embeddings with negative sampling.

Benefits of pre-training and localized context demonstrated.

Abstract

We introduce SpERT, an attention model for span-based joint entity and relation extraction. Our key contribution is a light-weight reasoning on BERT embeddings, which features entity recognition and filtering, as well as relation classification with a localized, marker-free context representation. The model is trained using strong within-sentence negative samples, which are efficiently extracted in a single BERT pass. These aspects facilitate a search over all spans in the sentence. In ablation studies, we demonstrate the benefits of pre-training, strong negative sampling and localized context. Our model outperforms prior work by up to 2.6% F1 score on several datasets for joint entity and relation extraction.