Relation Extraction as Open-book Examination: Retrieval-enhanced Prompt Tuning

TL;DR

This paper introduces a retrieval-enhanced prompt tuning method for relation extraction, framing it as an open-book exam, which improves generalization especially for rare or hard patterns in few-shot and supervised settings.

Contribution

It proposes a novel semiparametric paradigm combining parametric models with a retrieval datastore, achieving state-of-the-art results in relation extraction tasks.

Findings

Achieves state-of-the-art performance on benchmark datasets.

Effective in both supervised and few-shot relation extraction.

Enhances generalization to rare or hard relation patterns.

Abstract

Pre-trained language models have contributed significantly to relation extraction by demonstrating remarkable few-shot learning abilities. However, prompt tuning methods for relation extraction may still fail to generalize to those rare or hard patterns. Note that the previous parametric learning paradigm can be viewed as memorization regarding training data as a book and inference as the close-book test. Those long-tailed or hard patterns can hardly be memorized in parameters given few-shot instances. To this end, we regard RE as an open-book examination and propose a new semiparametric paradigm of retrieval-enhanced prompt tuning for relation extraction. We construct an open-book datastore for retrieval regarding prompt-based instance representations and corresponding relation labels as memorized key-value pairs. During inference, the model can infer relations by linearly interpolating the base output of PLM with the non-parametric nearest neighbor distribution over the datastore. In this way, our model not only infers relation through knowledge stored in the weights during training but also assists decision-making by unwinding and querying examples in the open-book datastore. Extensive experiments on benchmark datasets show that our method can achieve state-of-the-art in both standard supervised and few-shot settings. Code are available in https://github.com/zjunlp/PromptKG/tree/main/research/RetrievalRE.