Revisiting the Negative Data of Distantly Supervised Relation Extraction

TL;DR

This paper analyzes the negative data challenges in distantly supervised relation extraction, proposing a positive unlabeled learning framework and a pipeline method to improve sample efficiency and robustness.

Contribution

It introduces a novel formulation of relation extraction as a positive unlabeled learning problem and a pipeline approach called ReRe for better handling negative data issues.

Findings

ReRe outperforms existing methods in relation extraction tasks.

The approach remains effective even with many false positive samples.

Analysis clarifies the impact of negative data on relation extraction.

Abstract

Distantly supervision automatically generates plenty of training samples for relation extraction. However, it also incurs two major problems: noisy labels and imbalanced training data. Previous works focus more on reducing wrongly labeled relations (false positives) while few explore the missing relations that are caused by incompleteness of knowledge base (false negatives). Furthermore, the quantity of negative labels overwhelmingly surpasses the positive ones in previous problem formulations. In this paper, we first provide a thorough analysis of the above challenges caused by negative data. Next, we formulate the problem of relation extraction into as a positive unlabeled learning task to alleviate false negative problem. Thirdly, we propose a pipeline approach, dubbed \textsc{ReRe}, that performs sentence-level relation detection then subject/object extraction to achieve sample-efficient training. Experimental results show that the proposed method consistently outperforms existing approaches and remains excellent performance even learned with a large quantity of false positive samples.