Representation Learning for Weakly Supervised Relation Extraction

TL;DR

This paper introduces unsupervised pre-training models to learn rich text representations that enhance supervised relation extraction, especially when training data is limited, by combining these features with traditional hand-crafted features.

Contribution

The paper proposes novel unsupervised pre-training models for text representation to improve relation extraction performance with limited labeled data.

Findings

Pre-trained features improve relation classification accuracy.

Combining learned features with hand-crafted features enhances performance.

Significant gains observed on relations with scarce training data.

Abstract

Recent years have seen rapid development in Information Extraction, as well as its subtask, Relation Extraction. Relation Extraction is able to detect semantic relations between entities in sentences. Currently, many efficient approaches have been applied to relation extraction tasks. Supervised learning approaches especially have good performance. However, there are still many difficult challenges. One of the most serious problems is that manually labeled data is difficult to acquire. In most cases, limited data for supervised approaches equals lousy performance. Thus here, under the situation with only limited training data, we focus on how to improve the performance of our supervised baseline system with unsupervised pre-training. Feature is one of the key components in improving the supervised approaches. Traditional approaches usually apply hand-crafted features, which require expert knowledge and expensive human labor. However, this type of feature might suffer from data sparsity: when the training set size is small, the model parameters might be poorly estimated. In this thesis, we present several novel unsupervised pre-training models to learn the distributed text representation features, which are encoded with rich syntactic-semantic patterns of relation expressions. The experiments have demonstrated that this type of feature, combine with the traditional hand-crafted features, could improve the performance of the logistic classification model for relation extraction, especially on the classification of relations with only minor training instances.