Finding Influential Instances for Distantly Supervised Relation Extraction

TL;DR

This paper introduces REIF, a model-agnostic influence function-based method for selecting beneficial instances in distantly supervised relation extraction, improving stability, interpretability, and performance over existing black-box approaches.

Contribution

The work proposes a novel influence function-based sampling method for distantly supervised relation extraction, offering interpretability and computational efficiency improvements.

Findings

REIF outperforms baselines with complex architectures.

REIF provides interpretable instance selection.

The influence sampling algorithm reduces complexity to O(1).

Abstract

Distant supervision (DS) is a strong way to expand the datasets for enhancing relation extraction (RE) models but often suffers from high label noise. Current works based on attention, reinforcement learning, or GAN are black-box models so they neither provide meaningful interpretation of sample selection in DS nor stability on different domains. On the contrary, this work proposes a novel model-agnostic instance sampling method for DS by influence function (IF), namely REIF. Our method identifies favorable/unfavorable instances in the bag based on IF, then does dynamic instance sampling. We design a fast influence sampling algorithm that reduces the computational complexity from $\mathcal{O}(mn)$ to $\mathcal{O}(1)$, with analyzing its robustness on the selected sampling function. Experiments show that by simply sampling the favorable instances during training, REIF is able to win over a series of baselines that have complicated architectures. We also demonstrate that REIF can support interpretable instance selection.