Fine-grained Contrastive Learning for Relation Extraction

TL;DR

This paper introduces FineCL, a novel contrastive learning approach for relation extraction that accounts for the varying reliability of silver labels, leading to improved representation quality and better performance on benchmarks.

Contribution

FineCL leverages a new learning order denoising method to identify and emphasize more reliable silver labels during contrastive pre-training for relation extraction.

Findings

FineCL outperforms state-of-the-art methods on multiple RE benchmarks.

Learning order correlates with label accuracy, guiding label weighting.

Increased weights on accurate labels improve relation representation quality.

Abstract

Recent relation extraction (RE) works have shown encouraging improvements by conducting contrastive learning on silver labels generated by distant supervision before fine-tuning on gold labels. Existing methods typically assume all these silver labels are accurate and treat them equally; however, distant supervision is inevitably noisy -- some silver labels are more reliable than others. In this paper, we propose fine-grained contrastive learning (FineCL) for RE, which leverages fine-grained information about which silver labels are and are not noisy to improve the quality of learned relationship representations for RE. We first assess the quality of silver labels via a simple and automatic approach we call "learning order denoising," where we train a language model to learn these relations and record the order of learned training instances. We show that learning order largely corresponds to label accuracy -- early-learned silver labels have, on average, more accurate labels than later-learned silver labels. Then, during pre-training, we increase the weights of accurate labels within a novel contrastive learning objective. Experiments on several RE benchmarks show that FineCL makes consistent and significant performance gains over state-of-the-art methods.