Deep Residual Learning for Weakly-Supervised Relation Extraction

TL;DR

This paper explores the application of deep residual learning (ResNet) to noisy relation extraction in natural language processing, demonstrating significant performance improvements even with relatively shallow networks.

Contribution

It introduces a novel CNN architecture with residual learning for relation extraction and shows its effectiveness on noisy, distantly-supervised data.

Findings

Residual learning improves relation extraction performance.

Even 9-layer CNNs benefit from residual connections.

ResNet achieves state-of-the-art results in noisy NLP tasks.

Abstract

Deep residual learning (ResNet) is a new method for training very deep neural networks using identity map-ping for shortcut connections. ResNet has won the ImageNet ILSVRC 2015 classification task, and achieved state-of-the-art performances in many computer vision tasks. However, the effect of residual learning on noisy natural language processing tasks is still not well understood. In this paper, we design a novel convolutional neural network (CNN) with residual learning, and investigate its impacts on the task of distantly supervised noisy relation extraction. In contradictory to popular beliefs that ResNet only works well for very deep networks, we found that even with 9 layers of CNNs, using identity mapping could significantly improve the performance for distantly-supervised relation extraction.