Learning with Silver Standard Data for Zero-shot Relation Extraction

TL;DR

This paper introduces a method to leverage silver standard data for zero-shot relation extraction by detecting clean data and fine-tuning models, significantly improving performance over baselines.

Contribution

It proposes a class-aware clean data detection approach to utilize silver standard data effectively for zero-shot RE, enhancing existing methods.

Findings

Outperforms baseline by 12% on TACRED

Outperforms baseline by 11% on Wiki80

Further improvements with additional silver data

Abstract

The superior performance of supervised relation extraction (RE) methods heavily relies on a large amount of gold standard data. Recent zero-shot relation extraction methods converted the RE task to other NLP tasks and used off-the-shelf models of these NLP tasks to directly perform inference on the test data without using a large amount of RE annotation data. A potentially valuable by-product of these methods is the large-scale silver standard data. However, there is no further investigation on the use of potentially valuable silver standard data. In this paper, we propose to first detect a small amount of clean data from silver standard data and then use the selected clean data to finetune the pretrained model. We then use the finetuned model to infer relation types. We also propose a class-aware clean data detection module to consider class information when selecting clean data. The experimental results show that our method can outperform the baseline by 12% and 11% on TACRED and Wiki80 dataset in the zero-shot RE task. By using extra silver standard data of different distributions, the performance can be further improved.