Contrastive Triple Extraction with Generative Transformer

TL;DR

This paper presents a novel contrastive generative transformer model for triple extraction that improves faithfulness and performance in information extraction tasks, addressing limitations of previous sequence generation methods.

Contribution

The paper introduces a contrastive training mechanism and new attention and calibration techniques to enhance the accuracy and faithfulness of triple extraction with a generative transformer.

Findings

Outperforms baseline models on NYT, WebNLG, and MIE datasets.

Achieves higher precision and recall in triple extraction.

Demonstrates effectiveness of contrastive training in sequence generation.

Abstract

Triple extraction is an essential task in information extraction for natural language processing and knowledge graph construction. In this paper, we revisit the end-to-end triple extraction task for sequence generation. Since generative triple extraction may struggle to capture long-term dependencies and generate unfaithful triples, we introduce a novel model, contrastive triple extraction with a generative transformer. Specifically, we introduce a single shared transformer module for encoder-decoder-based generation. To generate faithful results, we propose a novel triplet contrastive training object. Moreover, we introduce two mechanisms to further improve model performance (i.e., batch-wise dynamic attention-masking and triple-wise calibration). Experimental results on three datasets (i.e., NYT, WebNLG, and MIE) show that our approach achieves better performance than that of baselines.