A Positive-Unlabeled Metric Learning Framework for Document-Level Relation Extraction with Incomplete Labeling

TL;DR

This paper introduces P3M, a positive-unlabeled metric learning framework for document-level relation extraction that improves performance with incomplete labels and achieves state-of-the-art results.

Contribution

The paper proposes a novel metric learning approach with positive augmentation and mixup techniques tailored for incomplete labeling in document-level RE.

Findings

P3M improves F1 score by 4-10 points in incomplete labeling scenarios.

P3M achieves state-of-the-art results in fully labeled scenarios.

P3M demonstrates robustness to prior estimation bias.

Abstract

The goal of document-level relation extraction (RE) is to identify relations between entities that span multiple sentences. Recently, incomplete labeling in document-level RE has received increasing attention, and some studies have used methods such as positive-unlabeled learning to tackle this issue, but there is still a lot of room for improvement. Motivated by this, we propose a positive-augmentation and positive-mixup positive-unlabeled metric learning framework (P3M). Specifically, we formulate document-level RE as a metric learning problem. We aim to pull the distance closer between entity pair embedding and their corresponding relation embedding, while pushing it farther away from the none-class relation embedding. Additionally, we adapt the positive-unlabeled learning to this loss objective. In order to improve the generalizability of the model, we use dropout to augment positive samples and propose a positive-none-class mixup method. Extensive experiments show that P3M improves the F1 score by approximately 4-10 points in document-level RE with incomplete labeling, and achieves state-of-the-art results in fully labeled scenarios. Furthermore, P3M has also demonstrated robustness to prior estimation bias in incomplete labeled scenarios.