FREDA: Flexible Relation Extraction Data Annotation

TL;DR

FREDA introduces a flexible annotation method that rapidly produces high-quality relation extraction datasets, enabling effective training of neural models with good generalization across datasets.

Contribution

The paper presents a novel annotation approach that accelerates dataset creation for relation extraction without sacrificing quality.

Findings

Annotated 10,022 sentences for 19 relations efficiently

Neural models trained on FREDA datasets perform well and generalize

High-quality datasets can be produced quickly for relation extraction

Abstract

To effectively train accurate Relation Extraction models, sufficient and properly labeled data is required. Adequately labeled data is difficult to obtain and annotating such data is a tricky undertaking. Previous works have shown that either accuracy has to be sacrificed or the task is extremely time-consuming, if done accurately. We are proposing an approach in order to produce high-quality datasets for the task of Relation Extraction quickly. Neural models, trained to do Relation Extraction on the created datasets, achieve very good results and generalize well to other datasets. In our study, we were able to annotate 10,022 sentences for 19 relations in a reasonable amount of time, and trained a commonly used baseline model for each relation.