Label Verbalization and Entailment for Effective Zero- and Few-Shot Relation Extraction

TL;DR

This paper reformulates relation extraction as an entailment task using verbalizations, enabling effective zero- and few-shot learning with minimal annotation, achieving competitive results on TACRED with significantly less data.

Contribution

It introduces a novel entailment-based approach for relation extraction that requires minimal manual effort and achieves state-of-the-art results in low-data settings.

Findings

Zero-shot system attains 63% F1 on TACRED.

Few-shot system with 16 examples surpasses supervised models by 17%.

Performance improves with larger entailment models, nearing state-of-the-art.

Abstract

Relation extraction systems require large amounts of labeled examples which are costly to annotate. In this work we reformulate relation extraction as an entailment task, with simple, hand-made, verbalizations of relations produced in less than 15 min per relation. The system relies on a pretrained textual entailment engine which is run as-is (no training examples, zero-shot) or further fine-tuned on labeled examples (few-shot or fully trained). In our experiments on TACRED we attain 63% F1 zero-shot, 69% with 16 examples per relation (17% points better than the best supervised system on the same conditions), and only 4 points short to the state-of-the-art (which uses 20 times more training data). We also show that the performance can be improved significantly with larger entailment models, up to 12 points in zero-shot, allowing to report the best results to date on TACRED when fully trained. The analysis shows that our few-shot systems are specially effective when discriminating between relations, and that the performance difference in low data regimes comes mainly from identifying no-relation cases.