Generative Meta-Learning for Zero-Shot Relation Triplet Extraction

TL;DR

This paper introduces a novel generative meta-learning framework combined with bi-level optimization to improve zero-shot relation triplet extraction, enabling models to better generalize to unseen relation types.

Contribution

It proposes a new generative meta-learning approach with bi-level optimization to enhance zero-shot relation extraction capabilities.

Findings

Outperforms existing methods on ZeroRTE tasks

Demonstrates improved generalization to unseen relation types

Provides a flexible framework adaptable to different meta-learning categories

Abstract

Zero-shot Relation Triplet Extraction (ZeroRTE) aims to extract relation triplets from texts containing unseen relation types. This capability benefits various downstream information retrieval (IR) tasks. The primary challenge lies in enabling models to generalize effectively to unseen relation categories. Existing approaches typically leverage the knowledge embedded in pre-trained language models to accomplish the generalization process. However, these methods focus solely on fitting the training data during training, without specifically improving the model's generalization performance, resulting in limited generalization capability. For this reason, we explore the integration of bi-level optimization (BLO) with pre-trained language models for learning generalized knowledge directly from the training data, and propose a generative meta-learning framework which exploits the `learning-to-learn' ability of meta-learning to boost the generalization capability of generative models. Specifically, we introduce a BLO approach that simultaneously addresses data fitting and generalization. This is achieved by constructing an upper-level loss to focus on generalization and a lower-level loss to ensure accurate data fitting. Building on this, we subsequently develop three generative meta-learning methods, each tailored to a distinct category of meta-learning. Extensive experimental results demonstrate that our framework performs well on the ZeroRTE task. Our code is available at https://github.com/leeworry/TGM-MetaLearning.