Gradient Imitation Reinforcement Learning for Low Resource Relation Extraction

TL;DR

This paper introduces GradLRE, a novel reinforcement learning approach that improves low-resource relation extraction by imitating gradient descent directions, effectively utilizing limited labeled data and data augmentation.

Contribution

The paper proposes a Gradient Imitation Reinforcement Learning framework for low-resource relation extraction, addressing feedback limitations and incorporating data augmentation for scenarios with no unlabeled data.

Findings

GradLRE outperforms baseline methods on two public datasets.

The method effectively handles both with and without unlabeled data scenarios.

Experimental results show significant improvement in low-resource settings.

Abstract

Low-resource Relation Extraction (LRE) aims to extract relation facts from limited labeled corpora when human annotation is scarce. Existing works either utilize self-training scheme to generate pseudo labels that will cause the gradual drift problem, or leverage meta-learning scheme which does not solicit feedback explicitly. To alleviate selection bias due to the lack of feedback loops in existing LRE learning paradigms, we developed a Gradient Imitation Reinforcement Learning method to encourage pseudo label data to imitate the gradient descent direction on labeled data and bootstrap its optimization capability through trial and error. We also propose a framework called GradLRE, which handles two major scenarios in low-resource relation extraction. Besides the scenario where unlabeled data is sufficient, GradLRE handles the situation where no unlabeled data is available, by exploiting a contextualized augmentation method to generate data. Experimental results on two public datasets demonstrate the effectiveness of GradLRE on low resource relation extraction when comparing with baselines.