Prompt-Contrastive Learning for Zero-Shot Relation Extraction

TL;DR

This paper introduces a new method for zero-shot relation extraction using prompt-contrastive learning to improve performance by leveraging pre-trained language models.

Contribution

The novel PCRE method uses prompt-contrastive learning to enhance zero-shot relation extraction by exploiting relational knowledge from pre-trained models.

Findings

PCRE outperforms state-of-the-art baselines in zero-shot relation extraction.

The method is robust across different datasets and varying numbers of seen relations and training instances.

Abstract

Relation extraction serves as an essential task for knowledge acquisition and management, defined as determining the relation between two annotated entities from a piece of text. Over recent years, zero-shot learning has been introduced to train relation extraction models due to the expensive cost of incessantly annotating emerging relations. Current methods endeavor to transfer knowledge of seen relations into predictions of unseen relations by conducting relation extraction through different tasks. Nonetheless, the divergence in task formulations prevents relation extraction models from acquiring informative semantic representations, resulting in inferior performance. In this paper, we strive to exploit the relational knowledge contained in pre-trained language models, which may generate enlightening information for the representation of unseen relations from seen relations. To this end, we investigate a Prompt-Contrastive learning perspective for Relation Extraction under a zero-shot setting, namely PCRE. To be specific, based on leveraging semantic knowledge from pre-trained language models with prompt tuning, we augment each instance with different prompt templates to construct two views for an instance-level contrastive objective. Additionally, we devise an instance-description contrastive objective to elicit relational knowledge from relation descriptions. With joint optimization, the relation extraction model can learn how to separate relations. The experimental results show our PCRE method outperforms state-of-the-art baselines in zero-shot relation extraction. The further extensive analysis verifies that our proposal is robust in different datasets, the number of seen relations, and the number of training instances.

Figures