Dual Supervision Framework for Relation Extraction with Distant Supervision and Human Annotation

TL;DR

This paper introduces a dual supervision framework for relation extraction that combines human-annotated and distantly supervised data using separate networks and an disagreement penalty to improve accuracy.

Contribution

It proposes a novel dual supervision approach with separate prediction networks and an adaptive bias assessment to effectively leverage both data types in relation extraction.

Findings

Improved relation extraction accuracy on sentence and document levels.

Effective use of both human annotation and distant supervision data.

Demonstrated robustness of the framework across different RE tasks.

Abstract

Relation extraction (RE) has been extensively studied due to its importance in real-world applications such as knowledge base construction and question answering. Most of the existing works train the models on either distantly supervised data or human-annotated data. To take advantage of the high accuracy of human annotation and the cheap cost of distant supervision, we propose the dual supervision framework which effectively utilizes both types of data. However, simply combining the two types of data to train a RE model may decrease the prediction accuracy since distant supervision has labeling bias. We employ two separate prediction networks HA-Net and DS-Net to predict the labels by human annotation and distant supervision, respectively, to prevent the degradation of accuracy by the incorrect labeling of distant supervision. Furthermore, we propose an additional loss term called disagreement penalty to enable HA-Net to learn from distantly supervised labels. In addition, we exploit additional networks to adaptively assess the labeling bias by considering contextual information. Our performance study on sentence-level and document-level REs confirms the effectiveness of the dual supervision framework.