Gradient Imitation Reinforcement Learning for General Low-Resource Information Extraction

TL;DR

This paper introduces GIRL, a reinforcement learning approach that improves low-resource information extraction by guiding pseudo-labeled data to imitate the gradient descent of labeled data, enhancing learning without extensive annotations.

Contribution

The paper proposes a novel Gradient Imitation Reinforcement Learning method that addresses confirmation bias in low-resource IE tasks by encouraging pseudo-labels to mimic gradient directions from labeled data.

Findings

GIRL improves semi-supervised and few-shot IE performance.

The method effectively reduces confirmation bias in low-resource settings.

GIRL is applicable across multiple IE sub-tasks.

Abstract

Information Extraction (IE) aims to extract structured information from heterogeneous sources. IE from natural language texts include sub-tasks such as Named Entity Recognition (NER), Relation Extraction (RE), and Event Extraction (EE). Most IE systems require comprehensive understandings of sentence structure, implied semantics, and domain knowledge to perform well; thus, IE tasks always need adequate external resources and annotations. However, it takes time and effort to obtain more human annotations. Low-Resource Information Extraction (LRIE) strives to use unsupervised data, reducing the required resources and human annotation. In practice, existing systems either utilize self-training schemes to generate pseudo labels that will cause the gradual drift problem, or leverage consistency regularization methods which inevitably possess confirmation bias. To alleviate confirmation bias due to the lack of feedback loops in existing LRIE learning paradigms, we develop a Gradient Imitation Reinforcement Learning (GIRL) method to encourage pseudo-labeled data to imitate the gradient descent direction on labeled data, which can force pseudo-labeled data to achieve better optimization capabilities similar to labeled data. Based on how well the pseudo-labeled data imitates the instructive gradient descent direction obtained from labeled data, we design a reward to quantify the imitation process and bootstrap the optimization capability of pseudo-labeled data through trial and error. In addition to learning paradigms, GIRL is not limited to specific sub-tasks, and we leverage GIRL to solve all IE sub-tasks (named entity recognition, relation extraction, and event extraction) in low-resource settings (semi-supervised IE and few-shot IE).